2024-08-08T20:39:41.2209028Z Current runner version: '2.319.0' 2024-08-08T20:39:41.2218449Z Runner name: 'i-09af04530f81e3933' 2024-08-08T20:39:41.2219539Z Runner group name: 'Default' 2024-08-08T20:39:41.2220676Z Machine name: 'ip-10-0-49-49' 2024-08-08T20:39:41.2245321Z ##[group]GITHUB_TOKEN Permissions 2024-08-08T20:39:41.2248140Z Actions: read 2024-08-08T20:39:41.2248775Z Attestations: read 2024-08-08T20:39:41.2249540Z Checks: read 2024-08-08T20:39:41.2250117Z Contents: read 2024-08-08T20:39:41.2250671Z Deployments: read 2024-08-08T20:39:41.2251445Z Discussions: read 2024-08-08T20:39:41.2252052Z Issues: read 2024-08-08T20:39:41.2252582Z Metadata: read 2024-08-08T20:39:41.2253261Z Packages: read 2024-08-08T20:39:41.2253869Z Pages: read 2024-08-08T20:39:41.2254412Z PullRequests: read 2024-08-08T20:39:41.2255143Z RepositoryProjects: read 2024-08-08T20:39:41.2255866Z SecurityEvents: read 2024-08-08T20:39:41.2256457Z Statuses: read 2024-08-08T20:39:41.2257126Z ##[endgroup] 2024-08-08T20:39:41.2262264Z Secret source: Actions 2024-08-08T20:39:41.2263506Z Prepare workflow directory 2024-08-08T20:39:41.7532769Z Prepare all required actions 2024-08-08T20:39:41.7786045Z Getting action download info 2024-08-08T20:39:41.9763682Z Download action repository 'pytorch/test-infra@main' (SHA:0b5c20199b94c3b12545de0d354b54a297b0e061) 2024-08-08T20:39:42.4297699Z Download action repository 'pytorch/pytorch@main' (SHA:7bd0732cbdca4ae572a8b4de2017cdfdff158d55) 2024-08-08T20:39:46.5054451Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-08T20:39:46.6636277Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-08T20:39:47.0041186Z Getting action download info 2024-08-08T20:39:47.0967332Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-08T20:39:47.3060323Z Getting action download info 2024-08-08T20:39:47.4026307Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-08T20:39:47.5465754Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/trunk/132710 (b9d86fa89636e301796d4201f36d86c73f6e49bc) 2024-08-08T20:39:47.5469415Z ##[group] Inputs 2024-08-08T20:39:47.5470209Z build-environment: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build 2024-08-08T20:39:47.5475028Z test-matrix: {"include": [{"config": "nogpu_AVX512", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "nogpu_NO_AVX2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}]} 2024-08-08T20:39:47.5480314Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:39:47.5481653Z sync-tag: 2024-08-08T20:39:47.5482866Z timeout-minutes: 240 2024-08-08T20:39:47.5483306Z use-gha: 2024-08-08T20:39:47.5483693Z dashboard-tag: 2024-08-08T20:39:47.5484036Z s3-bucket: gha-artifacts 2024-08-08T20:39:47.5484484Z aws-role-to-assume: 2024-08-08T20:39:47.5484887Z ##[endgroup] 2024-08-08T20:39:47.5486481Z Complete job name: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build-test / test (default, 4, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-08T20:39:47.6343754Z A job started hook has been configured by the self-hosted runner administrator 2024-08-08T20:39:47.6546583Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-08T20:39:47.6559518Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:39:47.6560209Z ##[endgroup] 2024-08-08T20:39:49.4632187Z Runner Type: amz2023.linux.4xlarge.nvidia.gpu 2024-08-08T20:39:49.4632922Z Instance Type: g3.4xlarge 2024-08-08T20:39:49.4633806Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-08T20:39:49.4634538Z AMI ID: ami-06c68f701d8090592 2024-08-08T20:39:56.9255316Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-08T20:39:56.9256128Z with: 2024-08-08T20:39:56.9257014Z github-secret: *** 2024-08-08T20:39:56.9258218Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-08T20:39:56.9259847Z activate-with-label: false 2024-08-08T20:39:56.9260304Z label: with-ssh 2024-08-08T20:39:56.9260685Z remove-existing-keys: true 2024-08-08T20:39:56.9261164Z fail-silently: true 2024-08-08T20:39:56.9261556Z env: 2024-08-08T20:39:56.9261870Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:39:56.9262299Z ##[endgroup] 2024-08-08T20:39:57.0447911Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-08T20:39:57.0453590Z ciflow reference detected, attempting to extract PR number 2024-08-08T20:39:57.3443045Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-08T20:39:57.4099402Z No SSH keys found for user pytorch-bot[bot] 2024-08-08T20:39:57.4100137Z Grabbing public ssh keys from https://github.com/drisspg.keys 2024-08-08T20:39:57.4855916Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-08T20:39:57.4878320Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-08T20:39:57.4922415Z Login using: ssh ec2-user@ec2-54-145-38-174.compute-1.amazonaws.com 2024-08-08T20:39:57.4923348Z All testing is done inside the container, to start an interactive session run: 2024-08-08T20:39:57.4924491Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-08T20:39:57.5062931Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-08T20:39:57.5063652Z with: 2024-08-08T20:39:57.5064002Z submodules: recursive 2024-08-08T20:39:57.5064421Z fetch-depth: 0 2024-08-08T20:39:57.5064760Z env: 2024-08-08T20:39:57.5065097Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:39:57.5065522Z ##[endgroup] 2024-08-08T20:39:57.5330225Z ##[group]Run retry () { 2024-08-08T20:39:57.5330700Z retry () { 2024-08-08T20:39:57.5331326Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-08T20:39:57.5332027Z } 2024-08-08T20:39:57.5332391Z echo "${GITHUB_WORKSPACE}" 2024-08-08T20:39:57.5332916Z if [ -z "${NO_SUDO}" ]; then 2024-08-08T20:39:57.5333519Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-08T20:39:57.5334066Z else 2024-08-08T20:39:57.5334502Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-08T20:39:57.5335032Z fi 2024-08-08T20:39:57.5335458Z mkdir "${GITHUB_WORKSPACE}" 2024-08-08T20:39:57.5343722Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:39:57.5344378Z env: 2024-08-08T20:39:57.5344727Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:39:57.5345133Z NO_SUDO: 2024-08-08T20:39:57.5345473Z ##[endgroup] 2024-08-08T20:39:57.5377832Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-08T20:39:59.9382127Z ##[group]Run malfet/checkout@silent-checkout 2024-08-08T20:39:59.9382679Z with: 2024-08-08T20:39:59.9383054Z ref: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:39:59.9383601Z fetch-depth: 0 2024-08-08T20:39:59.9383983Z submodules: recursive 2024-08-08T20:39:59.9384384Z quiet-checkout: true 2024-08-08T20:39:59.9384814Z repository: pytorch/pytorch 2024-08-08T20:39:59.9385416Z token: *** 2024-08-08T20:39:59.9385753Z ssh-strict: true 2024-08-08T20:39:59.9386478Z persist-credentials: true 2024-08-08T20:39:59.9386920Z clean: true 2024-08-08T20:39:59.9387310Z sparse-checkout-cone-mode: true 2024-08-08T20:39:59.9387789Z lfs: false 2024-08-08T20:39:59.9388157Z set-safe-directory: true 2024-08-08T20:39:59.9388553Z env: 2024-08-08T20:39:59.9388881Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:39:59.9389309Z ##[endgroup] 2024-08-08T20:40:00.0756782Z Syncing repository: pytorch/pytorch 2024-08-08T20:40:00.0759409Z ##[group]Getting Git version info 2024-08-08T20:40:00.0760380Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-08T20:40:00.0761507Z [command]/usr/bin/git version 2024-08-08T20:40:00.0761943Z git version 2.40.1 2024-08-08T20:40:00.0772174Z ##[endgroup] 2024-08-08T20:40:00.0796566Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/afa8ade3-bb82-4fb6-b207-d73248dd70f7' before making global git config changes 2024-08-08T20:40:00.0798199Z Adding repository directory to the temporary git global config as a safe directory 2024-08-08T20:40:00.0803682Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-08T20:40:00.0855009Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-08T20:40:00.0862069Z ##[group]Initializing the repository 2024-08-08T20:40:00.0866295Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-08T20:40:00.0905246Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-08T20:40:00.0906547Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-08T20:40:00.0907535Z hint: of your new repositories, which will suppress this warning, call: 2024-08-08T20:40:00.0908245Z hint: 2024-08-08T20:40:00.0908839Z hint: git config --global init.defaultBranch 2024-08-08T20:40:00.0909404Z hint: 2024-08-08T20:40:00.0910065Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-08T20:40:00.0911251Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-08T20:40:00.0913039Z hint: 2024-08-08T20:40:00.0913524Z hint: git branch -m 2024-08-08T20:40:00.0914493Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-08T20:40:00.0922746Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-08T20:40:00.0960939Z ##[endgroup] 2024-08-08T20:40:00.0961659Z ##[group]Disabling automatic garbage collection 2024-08-08T20:40:00.0965526Z [command]/usr/bin/git config --local gc.auto 0 2024-08-08T20:40:00.1005371Z ##[endgroup] 2024-08-08T20:40:00.1005953Z ##[group]Setting up auth 2024-08-08T20:40:00.1015540Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-08T20:40:00.1052612Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-08T20:40:00.1393340Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-08T20:40:00.1434987Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-08T20:40:00.1770223Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-08T20:40:00.1830147Z ##[endgroup] 2024-08-08T20:40:00.1831527Z ##[group]Fetching the repository 2024-08-08T20:40:00.1842146Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-08T20:40:02.6804911Z remote: Enumerating objects: 1008986 2024-08-08T20:40:02.6805595Z remote: Enumerating objects: 1011798, done. 2024-08-08T20:40:02.6807709Z remote: Counting objects: 0% (1/2812) 2024-08-08T20:40:02.6808911Z remote: Counting objects: 1% (29/2812) 2024-08-08T20:40:02.6811309Z remote: Counting objects: 2% (57/2812) 2024-08-08T20:40:02.6812923Z remote: Counting objects: 3% (85/2812) 2024-08-08T20:40:02.6814400Z remote: Counting objects: 4% (113/2812) 2024-08-08T20:40:02.6816994Z remote: Counting objects: 5% (141/2812) 2024-08-08T20:40:02.6818526Z remote: Counting objects: 6% (169/2812) 2024-08-08T20:40:02.6819543Z remote: Counting objects: 7% (197/2812) 2024-08-08T20:40:02.6820719Z remote: Counting objects: 8% (225/2812) 2024-08-08T20:40:02.6821368Z remote: Counting objects: 9% (254/2812) 2024-08-08T20:40:02.6822091Z remote: Counting objects: 10% (282/2812) 2024-08-08T20:40:02.6822755Z remote: Counting objects: 11% (310/2812) 2024-08-08T20:40:02.6824123Z remote: Counting objects: 12% (338/2812) 2024-08-08T20:40:02.6824790Z remote: Counting objects: 13% (366/2812) 2024-08-08T20:40:02.6829197Z remote: Counting objects: 14% (394/2812) 2024-08-08T20:40:02.6831508Z remote: Counting objects: 15% (422/2812) 2024-08-08T20:40:02.6832204Z remote: Counting objects: 16% (450/2812) 2024-08-08T20:40:02.6833209Z remote: Counting objects: 17% (479/2812) 2024-08-08T20:40:02.6833813Z remote: Counting objects: 18% (507/2812) 2024-08-08T20:40:02.6834448Z remote: Counting objects: 19% (535/2812) 2024-08-08T20:40:02.6834972Z remote: Counting objects: 20% (563/2812) 2024-08-08T20:40:02.6835644Z remote: Counting objects: 21% (591/2812) 2024-08-08T20:40:02.6836250Z remote: Counting objects: 22% (619/2812) 2024-08-08T20:40:02.6836933Z remote: Counting objects: 23% (647/2812) 2024-08-08T20:40:02.6838293Z remote: Counting objects: 24% (675/2812) 2024-08-08T20:40:02.6838997Z remote: Counting objects: 25% (703/2812) 2024-08-08T20:40:02.6839640Z remote: Counting objects: 26% (732/2812) 2024-08-08T20:40:02.6840263Z remote: Counting objects: 27% (760/2812) 2024-08-08T20:40:02.6841134Z remote: Counting objects: 28% (788/2812) 2024-08-08T20:40:02.6841725Z remote: Counting objects: 29% (816/2812) 2024-08-08T20:40:02.6842428Z remote: Counting objects: 30% (844/2812) 2024-08-08T20:40:02.6843314Z remote: Counting objects: 31% (872/2812) 2024-08-08T20:40:02.6844649Z remote: Counting objects: 32% (900/2812) 2024-08-08T20:40:02.6846050Z remote: Counting objects: 33% (928/2812) 2024-08-08T20:40:02.6846664Z remote: Counting objects: 34% (957/2812) 2024-08-08T20:40:02.6847342Z remote: Counting objects: 35% (985/2812) 2024-08-08T20:40:02.6848784Z remote: Counting objects: 36% (1013/2812) 2024-08-08T20:40:02.6849545Z remote: Counting objects: 37% (1041/2812) 2024-08-08T20:40:02.6850382Z remote: Counting objects: 38% (1069/2812) 2024-08-08T20:40:02.6851052Z remote: Counting objects: 39% (1097/2812) 2024-08-08T20:40:02.6852167Z remote: Counting objects: 40% (1125/2812) 2024-08-08T20:40:02.6853168Z remote: Counting objects: 41% (1153/2812) 2024-08-08T20:40:02.6854051Z remote: Counting objects: 42% (1182/2812) 2024-08-08T20:40:02.6854906Z remote: Counting objects: 43% (1210/2812) 2024-08-08T20:40:02.6855439Z remote: Counting objects: 44% (1238/2812) 2024-08-08T20:40:02.6856244Z remote: Counting objects: 45% (1266/2812) 2024-08-08T20:40:02.6857024Z remote: Counting objects: 46% (1294/2812) 2024-08-08T20:40:02.6857651Z remote: Counting objects: 47% (1322/2812) 2024-08-08T20:40:02.6858308Z remote: Counting objects: 48% (1350/2812) 2024-08-08T20:40:02.6859378Z remote: Counting objects: 49% (1378/2812) 2024-08-08T20:40:02.6860057Z remote: Counting objects: 50% (1406/2812) 2024-08-08T20:40:02.6860618Z remote: Counting objects: 51% (1435/2812) 2024-08-08T20:40:02.6861481Z remote: Counting objects: 52% (1463/2812) 2024-08-08T20:40:02.6862135Z remote: Counting objects: 53% (1491/2812) 2024-08-08T20:40:02.6862760Z remote: Counting objects: 54% (1519/2812) 2024-08-08T20:40:02.6863408Z remote: Counting objects: 55% (1547/2812) 2024-08-08T20:40:02.6864050Z remote: Counting objects: 56% (1575/2812) 2024-08-08T20:40:02.6864676Z remote: Counting objects: 57% (1603/2812) 2024-08-08T20:40:02.6865318Z remote: Counting objects: 58% (1631/2812) 2024-08-08T20:40:02.6865962Z remote: Counting objects: 59% (1660/2812) 2024-08-08T20:40:02.6866589Z remote: Counting objects: 60% (1688/2812) 2024-08-08T20:40:02.6867235Z remote: Counting objects: 61% (1716/2812) 2024-08-08T20:40:02.6867876Z remote: Counting objects: 62% (1744/2812) 2024-08-08T20:40:02.6868537Z remote: Counting objects: 63% (1772/2812) 2024-08-08T20:40:02.6869166Z remote: Counting objects: 64% (1800/2812) 2024-08-08T20:40:02.6869820Z remote: Counting objects: 65% (1828/2812) 2024-08-08T20:40:02.6870475Z remote: Counting objects: 66% (1856/2812) 2024-08-08T20:40:02.6871110Z remote: Counting objects: 67% (1885/2812) 2024-08-08T20:40:02.6871760Z remote: Counting objects: 68% (1913/2812) 2024-08-08T20:40:02.6872408Z remote: Counting objects: 69% (1941/2812) 2024-08-08T20:40:02.6873057Z remote: Counting objects: 70% (1969/2812) 2024-08-08T20:40:02.6873684Z remote: Counting objects: 71% (1997/2812) 2024-08-08T20:40:02.6874331Z remote: Counting objects: 72% (2025/2812) 2024-08-08T20:40:02.6874979Z remote: Counting objects: 73% (2053/2812) 2024-08-08T20:40:02.6875604Z remote: Counting objects: 74% (2081/2812) 2024-08-08T20:40:02.6876251Z remote: Counting objects: 75% (2109/2812) 2024-08-08T20:40:02.6876894Z remote: Counting objects: 76% (2138/2812) 2024-08-08T20:40:02.6877518Z remote: Counting objects: 77% (2166/2812) 2024-08-08T20:40:02.6878178Z remote: Counting objects: 78% (2194/2812) 2024-08-08T20:40:02.6878950Z remote: Counting objects: 79% (2222/2812) 2024-08-08T20:40:02.6879600Z remote: Counting objects: 80% (2250/2812) 2024-08-08T20:40:02.6880246Z remote: Counting objects: 81% (2278/2812) 2024-08-08T20:40:02.6880889Z remote: Counting objects: 82% (2306/2812) 2024-08-08T20:40:02.6881561Z remote: Counting objects: 83% (2334/2812) 2024-08-08T20:40:02.6882190Z remote: Counting objects: 84% (2363/2812) 2024-08-08T20:40:02.6882834Z remote: Counting objects: 85% (2391/2812) 2024-08-08T20:40:02.6883482Z remote: Counting objects: 86% (2419/2812) 2024-08-08T20:40:02.6884106Z remote: Counting objects: 87% (2447/2812) 2024-08-08T20:40:02.6884747Z remote: Counting objects: 88% (2475/2812) 2024-08-08T20:40:02.6885392Z remote: Counting objects: 89% (2503/2812) 2024-08-08T20:40:02.6886018Z remote: Counting objects: 90% (2531/2812) 2024-08-08T20:40:02.6886674Z remote: Counting objects: 91% (2559/2812) 2024-08-08T20:40:02.6887325Z remote: Counting objects: 92% (2588/2812) 2024-08-08T20:40:02.6887952Z remote: Counting objects: 93% (2616/2812) 2024-08-08T20:40:02.6888594Z remote: Counting objects: 94% (2644/2812) 2024-08-08T20:40:02.6889241Z remote: Counting objects: 95% (2672/2812) 2024-08-08T20:40:02.6889866Z remote: Counting objects: 96% (2700/2812) 2024-08-08T20:40:02.6890509Z remote: Counting objects: 97% (2728/2812) 2024-08-08T20:40:02.6891150Z remote: Counting objects: 98% (2756/2812) 2024-08-08T20:40:02.6891774Z remote: Counting objects: 99% (2784/2812) 2024-08-08T20:40:02.6892421Z remote: Counting objects: 100% (2812/2812) 2024-08-08T20:40:02.6893108Z remote: Counting objects: 100% (2812/2812), done. 2024-08-08T20:40:02.7226297Z remote: Compressing objects: 0% (1/1456) 2024-08-08T20:40:02.7564129Z remote: Compressing objects: 1% (15/1456) 2024-08-08T20:40:02.7667124Z remote: Compressing objects: 2% (30/1456) 2024-08-08T20:40:02.8123057Z remote: Compressing objects: 3% (44/1456) 2024-08-08T20:40:02.9257905Z remote: Compressing objects: 4% (59/1456) 2024-08-08T20:40:03.1211747Z remote: Compressing objects: 5% (73/1456) 2024-08-08T20:40:03.2450857Z remote: Compressing objects: 6% (88/1456) 2024-08-08T20:40:03.3060245Z remote: Compressing objects: 7% (102/1456) 2024-08-08T20:40:03.3371158Z remote: Compressing objects: 8% (117/1456) 2024-08-08T20:40:03.3759051Z remote: Compressing objects: 9% (132/1456) 2024-08-08T20:40:03.4217322Z remote: Compressing objects: 10% (146/1456) 2024-08-08T20:40:03.4606948Z remote: Compressing objects: 11% (161/1456) 2024-08-08T20:40:03.4872850Z remote: Compressing objects: 12% (175/1456) 2024-08-08T20:40:03.5151693Z remote: Compressing objects: 13% (190/1456) 2024-08-08T20:40:03.5365881Z remote: Compressing objects: 14% (204/1456) 2024-08-08T20:40:03.5588011Z remote: Compressing objects: 15% (219/1456) 2024-08-08T20:40:03.5841323Z remote: Compressing objects: 16% (233/1456) 2024-08-08T20:40:03.5998020Z remote: Compressing objects: 17% (248/1456) 2024-08-08T20:40:03.6071943Z remote: Compressing objects: 18% (263/1456) 2024-08-08T20:40:03.6129469Z remote: Compressing objects: 19% (277/1456) 2024-08-08T20:40:03.6182963Z remote: Compressing objects: 20% (292/1456) 2024-08-08T20:40:03.6209128Z remote: Compressing objects: 21% (306/1456) 2024-08-08T20:40:03.6226192Z remote: Compressing objects: 22% (321/1456) 2024-08-08T20:40:03.6242117Z remote: Compressing objects: 23% (335/1456) 2024-08-08T20:40:03.6288581Z remote: Compressing objects: 24% (350/1456) 2024-08-08T20:40:03.6302052Z remote: Compressing objects: 25% (364/1456) 2024-08-08T20:40:03.6328136Z remote: Compressing objects: 26% (379/1456) 2024-08-08T20:40:03.6332920Z remote: Compressing objects: 27% (394/1456) 2024-08-08T20:40:03.6370097Z remote: Compressing objects: 28% (408/1456) 2024-08-08T20:40:03.6392351Z remote: Compressing objects: 29% (423/1456) 2024-08-08T20:40:03.6411104Z remote: Compressing objects: 30% (437/1456) 2024-08-08T20:40:03.6421768Z remote: Compressing objects: 31% (452/1456) 2024-08-08T20:40:03.6447427Z remote: Compressing objects: 32% (466/1456) 2024-08-08T20:40:03.6477408Z remote: Compressing objects: 33% (481/1456) 2024-08-08T20:40:03.6499870Z remote: Compressing objects: 34% (496/1456) 2024-08-08T20:40:03.6517597Z remote: Compressing objects: 35% (510/1456) 2024-08-08T20:40:03.6536423Z remote: Compressing objects: 36% (525/1456) 2024-08-08T20:40:03.6557907Z remote: Compressing objects: 37% (539/1456) 2024-08-08T20:40:03.6568902Z remote: Compressing objects: 38% (554/1456) 2024-08-08T20:40:03.6588106Z remote: Compressing objects: 39% (568/1456) 2024-08-08T20:40:03.6599228Z remote: Compressing objects: 40% (583/1456) 2024-08-08T20:40:03.6620908Z remote: Compressing objects: 41% (597/1456) 2024-08-08T20:40:03.6641045Z remote: Compressing objects: 42% (612/1456) 2024-08-08T20:40:03.6650934Z remote: Compressing objects: 43% (627/1456) 2024-08-08T20:40:03.6668338Z remote: Compressing objects: 44% (641/1456) 2024-08-08T20:40:03.6676930Z remote: Compressing objects: 45% (656/1456) 2024-08-08T20:40:03.6690864Z remote: Compressing objects: 46% (670/1456) 2024-08-08T20:40:03.6699471Z remote: Compressing objects: 47% (685/1456) 2024-08-08T20:40:03.6708456Z remote: Compressing objects: 48% (699/1456) 2024-08-08T20:40:03.6713767Z remote: Compressing objects: 49% (714/1456) 2024-08-08T20:40:03.6721678Z remote: Compressing objects: 50% (728/1456) 2024-08-08T20:40:03.6728149Z remote: Compressing objects: 51% (743/1456) 2024-08-08T20:40:03.6732374Z remote: Compressing objects: 52% (758/1456) 2024-08-08T20:40:03.6736833Z remote: Compressing objects: 53% (772/1456) 2024-08-08T20:40:03.6742962Z remote: Compressing objects: 54% (787/1456) 2024-08-08T20:40:03.6747494Z remote: Compressing objects: 55% (801/1456) 2024-08-08T20:40:03.6753150Z remote: Compressing objects: 56% (816/1456) 2024-08-08T20:40:03.6757160Z remote: Compressing objects: 57% (830/1456) 2024-08-08T20:40:03.6761362Z remote: Compressing objects: 58% (845/1456) 2024-08-08T20:40:03.6764222Z remote: Compressing objects: 59% (860/1456) 2024-08-08T20:40:03.6766846Z remote: Compressing objects: 60% (874/1456) 2024-08-08T20:40:03.6769404Z remote: Compressing objects: 61% (889/1456) 2024-08-08T20:40:03.6772068Z remote: Compressing objects: 62% (903/1456) 2024-08-08T20:40:03.6772740Z remote: Compressing objects: 63% (918/1456) 2024-08-08T20:40:03.6773718Z remote: Compressing objects: 64% (932/1456) 2024-08-08T20:40:03.6775104Z remote: Compressing objects: 65% (947/1456) 2024-08-08T20:40:03.6775929Z remote: Compressing objects: 66% (961/1456) 2024-08-08T20:40:03.6777061Z remote: Compressing objects: 67% (976/1456) 2024-08-08T20:40:03.6777636Z remote: Compressing objects: 68% (991/1456) 2024-08-08T20:40:03.6780114Z remote: Compressing objects: 69% (1005/1456) 2024-08-08T20:40:03.6786159Z remote: Compressing objects: 70% (1020/1456) 2024-08-08T20:40:03.6797345Z remote: Compressing objects: 71% (1034/1456) 2024-08-08T20:40:03.6805287Z remote: Compressing objects: 72% (1049/1456) 2024-08-08T20:40:03.6813576Z remote: Compressing objects: 73% (1063/1456) 2024-08-08T20:40:03.6815585Z remote: Compressing objects: 74% (1078/1456) 2024-08-08T20:40:03.6818499Z remote: Compressing objects: 75% (1092/1456) 2024-08-08T20:40:03.6821612Z remote: Compressing objects: 76% (1107/1456) 2024-08-08T20:40:03.6826872Z remote: Compressing objects: 77% (1122/1456) 2024-08-08T20:40:03.6830239Z remote: Compressing objects: 78% (1136/1456) 2024-08-08T20:40:03.6836945Z remote: Compressing objects: 79% (1151/1456) 2024-08-08T20:40:03.6839560Z remote: Compressing objects: 80% (1165/1456) 2024-08-08T20:40:03.6844155Z remote: Compressing objects: 81% (1180/1456) 2024-08-08T20:40:03.6847927Z remote: Compressing objects: 82% (1194/1456) 2024-08-08T20:40:03.6851837Z remote: Compressing objects: 83% (1209/1456) 2024-08-08T20:40:03.6854416Z remote: Compressing objects: 84% (1224/1456) 2024-08-08T20:40:03.6866196Z remote: Compressing objects: 85% (1238/1456) 2024-08-08T20:40:03.6867347Z remote: Compressing objects: 86% (1253/1456) 2024-08-08T20:40:03.6868524Z remote: Compressing objects: 86% (1259/1456) 2024-08-08T20:40:03.6869193Z remote: Compressing objects: 87% (1267/1456) 2024-08-08T20:40:03.6869880Z remote: Compressing objects: 88% (1282/1456) 2024-08-08T20:40:03.6881716Z remote: Compressing objects: 89% (1296/1456) 2024-08-08T20:40:03.6884308Z remote: Compressing objects: 90% (1311/1456) 2024-08-08T20:40:03.6885614Z remote: Compressing objects: 91% (1325/1456) 2024-08-08T20:40:03.6886280Z remote: Compressing objects: 92% (1340/1456) 2024-08-08T20:40:03.6887237Z remote: Compressing objects: 93% (1355/1456) 2024-08-08T20:40:03.6888021Z remote: Compressing objects: 94% (1369/1456) 2024-08-08T20:40:03.6888629Z remote: Compressing objects: 95% (1384/1456) 2024-08-08T20:40:03.6889952Z remote: Compressing objects: 96% (1398/1456) 2024-08-08T20:40:03.6890704Z remote: Compressing objects: 97% (1413/1456) 2024-08-08T20:40:03.6891388Z remote: Compressing objects: 98% (1427/1456) 2024-08-08T20:40:03.6892061Z remote: Compressing objects: 99% (1442/1456) 2024-08-08T20:40:03.6892978Z remote: Compressing objects: 100% (1456/1456) 2024-08-08T20:40:03.6893841Z remote: Compressing objects: 100% (1456/1456), done. 2024-08-08T20:40:31.5285261Z remote: Total 1011798 (delta 1871), reused 2070 (delta 1353), pack-reused 1008986 2024-08-08T20:40:58.1167480Z [command]/usr/bin/git rev-parse --verify --quiet b9d86fa89636e301796d4201f36d86c73f6e49bc^{object} 2024-08-08T20:40:58.1196513Z b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:40:58.1203173Z ##[endgroup] 2024-08-08T20:40:58.1203935Z ##[group]Determining the checkout info 2024-08-08T20:40:58.1205584Z ##[endgroup] 2024-08-08T20:40:58.1208686Z ##[group]Checking out the ref 2024-08-08T20:40:58.1210762Z [command]/usr/bin/git checkout --quiet --force b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:41:00.0260383Z ##[endgroup] 2024-08-08T20:41:00.0261973Z ##[group]Setting up auth for fetching submodules 2024-08-08T20:41:00.0268507Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-08T20:41:00.0332205Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-08T20:41:00.0369101Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-08T20:41:00.0405937Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-08T20:41:00.0440566Z ##[endgroup] 2024-08-08T20:41:00.0441103Z ##[group]Fetching submodules 2024-08-08T20:41:00.0446398Z [command]/usr/bin/git submodule sync --recursive 2024-08-08T20:41:00.0806973Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-08T20:41:00.1158323Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-08T20:41:00.1161690Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-08T20:41:00.1166235Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-08T20:41:00.1170042Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-08T20:41:00.1174374Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-08T20:41:00.1177854Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-08T20:41:00.1182245Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-08T20:41:00.1186767Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-08T20:41:00.1191242Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-08T20:41:00.1196042Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-08T20:41:00.1200655Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-08T20:41:00.1205482Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-08T20:41:00.1210440Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-08T20:41:00.1215715Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-08T20:41:00.1220719Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-08T20:41:00.1226139Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-08-08T20:41:00.1233978Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-08T20:41:00.1239502Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-08T20:41:00.1245184Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-08T20:41:00.1251169Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-08T20:41:00.1257142Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-08T20:41:00.1264076Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-08T20:41:00.1270021Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-08T20:41:00.1276162Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-08T20:41:00.1282442Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-08T20:41:00.1288834Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-08T20:41:00.1295833Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-08T20:41:00.1302287Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-08T20:41:00.1309147Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-08T20:41:00.1316189Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-08T20:41:00.1323647Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-08T20:41:00.1330430Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-08T20:41:00.1337695Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-08T20:41:00.1346888Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-08T20:41:00.1354596Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-08T20:41:00.1389044Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-08T20:41:00.5035830Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-08T20:41:00.6935854Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-08T20:41:00.9076260Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-08T20:41:01.1916114Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-08T20:41:03.5421164Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-08T20:41:17.5486052Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-08T20:41:18.0004558Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-08T20:41:18.5394266Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-08T20:41:19.2683396Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-08T20:41:20.7297847Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-08T20:41:23.1793991Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-08T20:41:30.3941942Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-08T20:41:32.0241100Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-08T20:41:33.9127971Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-08T20:41:35.3071723Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-08-08T20:41:35.4962543Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-08T20:41:35.9716634Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-08T20:41:36.3291563Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-08T20:41:37.6917432Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-08T20:41:38.1049216Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-08T20:41:38.3823684Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-08T20:41:40.2518556Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-08T20:41:41.0933239Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-08T20:41:41.7894363Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-08T20:41:49.1941446Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-08T20:41:51.6052319Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-08T20:41:56.8844313Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-08T20:41:57.5053330Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-08T20:42:08.1852480Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-08T20:42:08.4501129Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-08T20:42:08.6604977Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-08T20:42:09.7377972Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-08T20:42:10.0721839Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-08T20:42:10.7463824Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-08T20:42:11.2121413Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-08T20:42:11.2269566Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-08T20:42:11.2383661Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-08T20:42:11.2716303Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-08T20:42:11.3200115Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-08T20:42:12.6856536Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-08T20:42:12.7160951Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-08T20:42:12.7728642Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-08T20:42:12.9001571Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-08T20:42:12.9443846Z Submodule path 'third_party/cudnn_frontend': checked out '98ca4e1941fe3263f128f74f10063a3ea35c7019' 2024-08-08T20:42:13.6641990Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-08T20:42:13.9959291Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-08T20:42:14.1016152Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-08T20:42:14.1037590Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-08T20:42:14.1041072Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-08T20:42:14.1044900Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-08T20:42:14.1048825Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-08T20:42:14.1053184Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-08T20:42:14.1085188Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-08T20:42:15.3804642Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-08T20:42:16.0503028Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-08T20:42:18.4214496Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-08T20:42:19.7067290Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-08T20:42:20.0923540Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-08T20:42:20.2179331Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-08T20:42:20.7760753Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-08T20:42:20.8567052Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-08T20:42:20.8721657Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-08T20:42:21.0358757Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-08T20:42:21.0863293Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-08T20:42:21.0981268Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-08-08T20:42:21.1474960Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-08T20:42:21.1822375Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-08T20:42:21.2414481Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-08T20:42:21.2571889Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-08T20:42:21.2592256Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-08T20:42:21.2621708Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-08T20:42:36.7794590Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-08T20:42:36.8025431Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-08T20:42:36.9140033Z Submodule path 'third_party/kineto': checked out 'da2f2682cabaf95d601fa2a9b7e0979f84fe7667' 2024-08-08T20:42:36.9163548Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-08T20:42:36.9167265Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-08T20:42:36.9171449Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-08T20:42:36.9202736Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-08T20:42:37.5301754Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-08T20:42:38.8929470Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-08T20:42:40.3008853Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-08T20:42:40.3029943Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-08T20:42:40.3033874Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-08T20:42:40.3037865Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-08T20:42:40.3042185Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-08T20:42:40.3046642Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-08T20:42:40.3051145Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-08T20:42:40.3055442Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-08T20:42:40.3060324Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-08T20:42:40.3093090Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-08T20:42:41.3243668Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-08T20:42:41.7144821Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-08T20:42:43.4882391Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-08T20:42:43.9156600Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-08T20:42:44.7042291Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-08T20:42:46.0649892Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-08T20:42:53.5338452Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-08T20:42:54.0481213Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-08T20:42:54.0730550Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-08T20:42:54.1195061Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-08T20:42:54.1362530Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-08T20:42:54.1382302Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-08T20:42:54.1413824Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-08T20:42:54.4764434Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-08T20:42:54.4993332Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-08T20:42:54.5523942Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-08T20:42:54.6883679Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-08T20:42:54.7092346Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-08T20:42:54.7576378Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-08T20:42:54.8306250Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-08T20:42:54.8787988Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-08T20:42:54.9107327Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-08T20:42:55.0509530Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-08T20:42:55.5972423Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-08T20:42:55.6014873Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-08T20:42:55.6018761Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-08T20:42:55.6050955Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-08T20:42:56.1725556Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-08T20:42:57.2876908Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-08T20:42:57.3323742Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-08T20:42:57.4356769Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-08T20:42:57.4359366Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-08T20:42:57.4361690Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-08T20:42:57.4363627Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-08T20:42:57.4366039Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-08T20:42:57.4368533Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-08T20:42:57.4371093Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-08T20:42:57.4374758Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-08T20:42:57.4379324Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-08T20:42:57.4411963Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-08T20:42:57.9123578Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-08T20:42:59.1903565Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-08T20:42:59.5415216Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-08T20:43:06.9019667Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-08T20:43:07.2419787Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-08T20:43:07.4742217Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-08T20:43:07.8314642Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-08T20:43:15.4781018Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-08T20:43:15.5314552Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-08T20:43:15.5512867Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-08T20:43:15.6878795Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-08T20:43:15.7045264Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-08T20:43:15.7239614Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-08T20:43:15.7449006Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-08T20:43:15.7469743Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-08T20:43:15.7474069Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-08T20:43:15.7504018Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-08T20:43:18.0680871Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-08T20:43:19.6332823Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-08T20:43:19.6944881Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-08T20:43:20.4476164Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-08T20:43:20.4619180Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-08T20:43:20.8226832Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-08T20:43:20.8253202Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-08T20:43:20.8257159Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-08T20:43:20.8290521Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-08T20:43:21.4335361Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-08T20:43:22.7448096Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-08T20:43:22.8391420Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-08T20:43:22.8503929Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-08T20:43:22.8655218Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-08T20:43:22.9139759Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-08T20:43:22.9503181Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-08T20:43:23.0047912Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-08T20:43:23.0415099Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-08T20:43:23.0435956Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-08T20:43:23.0439624Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-08T20:43:23.0443424Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-08T20:43:23.0447371Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-08T20:43:23.0478510Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-08T20:43:24.3114475Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-08T20:43:24.5590979Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-08T20:43:25.9931690Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-08T20:43:27.2089241Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-08T20:43:27.2290069Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-08T20:43:27.3157554Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-08T20:43:27.3521834Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-08T20:43:27.3542032Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-08T20:43:27.3573067Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-08T20:43:27.6067809Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-08T20:43:27.6110111Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-08T20:43:27.6459040Z Entering 'android/libs/fbjni' 2024-08-08T20:43:27.6508933Z Entering 'third_party/FP16' 2024-08-08T20:43:27.6558076Z Entering 'third_party/FXdiv' 2024-08-08T20:43:27.6607035Z Entering 'third_party/NNPACK' 2024-08-08T20:43:27.6656416Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-08T20:43:27.6706832Z Entering 'third_party/XNNPACK' 2024-08-08T20:43:27.6774562Z Entering 'third_party/benchmark' 2024-08-08T20:43:27.6823420Z Entering 'third_party/cpp-httplib' 2024-08-08T20:43:27.6873630Z Entering 'third_party/cpuinfo' 2024-08-08T20:43:27.6922628Z Entering 'third_party/cudnn_frontend' 2024-08-08T20:43:27.6973153Z Entering 'third_party/cutlass' 2024-08-08T20:43:27.7031574Z Entering 'third_party/eigen' 2024-08-08T20:43:27.7085427Z Entering 'third_party/fbgemm' 2024-08-08T20:43:27.7134623Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-08T20:43:27.7184105Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-08T20:43:27.7231725Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-08T20:43:27.7288437Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-08T20:43:27.7336262Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-08T20:43:27.7387816Z Entering 'third_party/flatbuffers' 2024-08-08T20:43:27.7438781Z Entering 'third_party/fmt' 2024-08-08T20:43:27.7488146Z Entering 'third_party/foxi' 2024-08-08T20:43:27.7537682Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-08T20:43:27.7587335Z Entering 'third_party/gloo' 2024-08-08T20:43:27.7635955Z Entering 'third_party/googletest' 2024-08-08T20:43:27.7687678Z Entering 'third_party/ideep' 2024-08-08T20:43:27.7734811Z Entering 'third_party/ideep/mkl-dnn' 2024-08-08T20:43:27.7794013Z Entering 'third_party/ittapi' 2024-08-08T20:43:27.7842853Z Entering 'third_party/kineto' 2024-08-08T20:43:27.7893461Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-08T20:43:27.7941176Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-08T20:43:27.7992678Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-08T20:43:27.8041326Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-08T20:43:27.8091384Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-08T20:43:27.8139175Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-08T20:43:27.8191005Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-08T20:43:27.8240397Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-08T20:43:27.8289506Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-08T20:43:27.8339951Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-08T20:43:27.8391471Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-08T20:43:27.8440046Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-08T20:43:27.8491330Z Entering 'third_party/mimalloc' 2024-08-08T20:43:27.8539078Z Entering 'third_party/nccl/nccl' 2024-08-08T20:43:27.8589938Z Entering 'third_party/nlohmann' 2024-08-08T20:43:27.8639816Z Entering 'third_party/onnx' 2024-08-08T20:43:27.8708211Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-08T20:43:27.8758381Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-08T20:43:27.8811164Z Entering 'third_party/opentelemetry-cpp' 2024-08-08T20:43:27.8862008Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-08T20:43:27.8909269Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-08T20:43:27.8957382Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-08T20:43:27.9006060Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-08T20:43:27.9055529Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-08T20:43:27.9103908Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-08T20:43:27.9150823Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-08T20:43:27.9199723Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-08T20:43:27.9249816Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-08T20:43:27.9300545Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-08T20:43:27.9373200Z Entering 'third_party/pocketfft' 2024-08-08T20:43:27.9421239Z Entering 'third_party/protobuf' 2024-08-08T20:43:27.9474806Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-08T20:43:27.9522417Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-08T20:43:27.9572851Z Entering 'third_party/psimd' 2024-08-08T20:43:27.9621173Z Entering 'third_party/pthreadpool' 2024-08-08T20:43:27.9670791Z Entering 'third_party/pybind11' 2024-08-08T20:43:27.9719090Z Entering 'third_party/python-peachpy' 2024-08-08T20:43:27.9769486Z Entering 'third_party/sleef' 2024-08-08T20:43:27.9818034Z Entering 'third_party/tensorpipe' 2024-08-08T20:43:27.9868984Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-08T20:43:27.9917610Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-08T20:43:27.9966674Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-08T20:43:28.0015451Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-08T20:43:28.0064917Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-08T20:43:28.0130947Z ##[endgroup] 2024-08-08T20:43:28.0132246Z ##[group]Persisting credentials for submodules 2024-08-08T20:43:28.0138218Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-08T20:43:28.0483641Z Entering 'android/libs/fbjni' 2024-08-08T20:43:28.0547804Z Entering 'third_party/FP16' 2024-08-08T20:43:28.0612109Z Entering 'third_party/FXdiv' 2024-08-08T20:43:28.0675801Z Entering 'third_party/NNPACK' 2024-08-08T20:43:28.0740090Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-08T20:43:28.0804220Z Entering 'third_party/XNNPACK' 2024-08-08T20:43:28.0886177Z Entering 'third_party/benchmark' 2024-08-08T20:43:28.0950032Z Entering 'third_party/cpp-httplib' 2024-08-08T20:43:28.1014723Z Entering 'third_party/cpuinfo' 2024-08-08T20:43:28.1079182Z Entering 'third_party/cudnn_frontend' 2024-08-08T20:43:28.1143332Z Entering 'third_party/cutlass' 2024-08-08T20:43:28.1216931Z Entering 'third_party/eigen' 2024-08-08T20:43:28.1283732Z Entering 'third_party/fbgemm' 2024-08-08T20:43:28.1347701Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-08T20:43:28.1410537Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-08T20:43:28.1474415Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-08T20:43:28.1545994Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-08T20:43:28.1609079Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-08T20:43:28.1675280Z Entering 'third_party/flatbuffers' 2024-08-08T20:43:28.1741720Z Entering 'third_party/fmt' 2024-08-08T20:43:28.1807697Z Entering 'third_party/foxi' 2024-08-08T20:43:28.1871903Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-08T20:43:28.1935445Z Entering 'third_party/gloo' 2024-08-08T20:43:28.2000440Z Entering 'third_party/googletest' 2024-08-08T20:43:28.2063978Z Entering 'third_party/ideep' 2024-08-08T20:43:28.2125828Z Entering 'third_party/ideep/mkl-dnn' 2024-08-08T20:43:28.2198839Z Entering 'third_party/ittapi' 2024-08-08T20:43:28.2262914Z Entering 'third_party/kineto' 2024-08-08T20:43:28.2326332Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-08T20:43:28.2390033Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-08T20:43:28.2456013Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-08T20:43:28.2519876Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-08T20:43:28.2584239Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-08T20:43:28.2647259Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-08T20:43:28.2714111Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-08T20:43:28.2779490Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-08T20:43:28.2843197Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-08T20:43:28.2908611Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-08T20:43:28.2974467Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-08T20:43:28.3037201Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-08T20:43:28.3102320Z Entering 'third_party/mimalloc' 2024-08-08T20:43:28.3170882Z Entering 'third_party/nccl/nccl' 2024-08-08T20:43:28.3233807Z Entering 'third_party/nlohmann' 2024-08-08T20:43:28.3299558Z Entering 'third_party/onnx' 2024-08-08T20:43:28.3381656Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-08T20:43:28.3444200Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-08T20:43:28.3511279Z Entering 'third_party/opentelemetry-cpp' 2024-08-08T20:43:28.3576609Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-08T20:43:28.3639780Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-08T20:43:28.3703500Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-08T20:43:28.3767079Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-08T20:43:28.3831355Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-08T20:43:28.3894664Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-08T20:43:28.3957838Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-08T20:43:28.4019772Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-08T20:43:28.4085397Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-08T20:43:28.4150740Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-08T20:43:28.4237249Z Entering 'third_party/pocketfft' 2024-08-08T20:43:28.4300619Z Entering 'third_party/protobuf' 2024-08-08T20:43:28.4368723Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-08T20:43:28.4430158Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-08T20:43:28.4496262Z Entering 'third_party/psimd' 2024-08-08T20:43:28.4559424Z Entering 'third_party/pthreadpool' 2024-08-08T20:43:28.4622258Z Entering 'third_party/pybind11' 2024-08-08T20:43:28.4686613Z Entering 'third_party/python-peachpy' 2024-08-08T20:43:28.4749398Z Entering 'third_party/sleef' 2024-08-08T20:43:28.4812768Z Entering 'third_party/tensorpipe' 2024-08-08T20:43:28.4876174Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-08T20:43:28.4938384Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-08T20:43:28.5002553Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-08T20:43:28.5068548Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-08T20:43:28.5129375Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-08T20:43:28.5213309Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-08T20:43:28.5558773Z Entering 'android/libs/fbjni' 2024-08-08T20:43:28.5618219Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-08T20:43:28.5637744Z Entering 'third_party/FP16' 2024-08-08T20:43:28.5698933Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-08T20:43:28.5718248Z Entering 'third_party/FXdiv' 2024-08-08T20:43:28.5778940Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-08T20:43:28.5798143Z Entering 'third_party/NNPACK' 2024-08-08T20:43:28.5857974Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-08T20:43:28.5878065Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-08T20:43:28.5949653Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-08T20:43:28.5959126Z Entering 'third_party/XNNPACK' 2024-08-08T20:43:28.6018900Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-08T20:43:28.6057501Z Entering 'third_party/benchmark' 2024-08-08T20:43:28.6117783Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-08T20:43:28.6137000Z Entering 'third_party/cpp-httplib' 2024-08-08T20:43:28.6197218Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-08T20:43:28.6215953Z Entering 'third_party/cpuinfo' 2024-08-08T20:43:28.6278109Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-08T20:43:28.6297796Z Entering 'third_party/cudnn_frontend' 2024-08-08T20:43:28.6357771Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-08T20:43:28.6376832Z Entering 'third_party/cutlass' 2024-08-08T20:43:28.6436970Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-08T20:43:28.6465613Z Entering 'third_party/eigen' 2024-08-08T20:43:28.6524947Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-08T20:43:28.6547180Z Entering 'third_party/fbgemm' 2024-08-08T20:43:28.6608844Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-08T20:43:28.6628184Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-08T20:43:28.6689806Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-08T20:43:28.6707996Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-08T20:43:28.6767707Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-08T20:43:28.6786199Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-08T20:43:28.6845510Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-08T20:43:28.6871684Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-08T20:43:28.6930040Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-08T20:43:28.6948405Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-08T20:43:28.7008180Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-08T20:43:28.7028278Z Entering 'third_party/flatbuffers' 2024-08-08T20:43:28.7088214Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-08T20:43:28.7110265Z Entering 'third_party/fmt' 2024-08-08T20:43:28.7171141Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-08T20:43:28.7190351Z Entering 'third_party/foxi' 2024-08-08T20:43:28.7249544Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-08-08T20:43:28.7269937Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-08T20:43:28.7329531Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-08T20:43:28.7348406Z Entering 'third_party/gloo' 2024-08-08T20:43:28.7410516Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-08T20:43:28.7430031Z Entering 'third_party/googletest' 2024-08-08T20:43:28.7489861Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-08T20:43:28.7509245Z Entering 'third_party/ideep' 2024-08-08T20:43:28.7571013Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-08T20:43:28.7588735Z Entering 'third_party/ideep/mkl-dnn' 2024-08-08T20:43:28.7647217Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-08T20:43:28.7676081Z Entering 'third_party/ittapi' 2024-08-08T20:43:28.7735386Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-08T20:43:28.7754698Z Entering 'third_party/kineto' 2024-08-08T20:43:28.7817328Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-08T20:43:28.7836248Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-08T20:43:28.7897717Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-08T20:43:28.7915107Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-08T20:43:28.7978066Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-08T20:43:28.7997198Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-08T20:43:28.8057984Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-08T20:43:28.8077110Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-08T20:43:28.8137427Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-08T20:43:28.8156290Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-08T20:43:28.8218203Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-08T20:43:28.8235081Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-08T20:43:28.8298615Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-08T20:43:28.8318571Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-08T20:43:28.8380366Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-08T20:43:28.8398906Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-08T20:43:28.8459100Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-08T20:43:28.8478362Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-08T20:43:28.8538546Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-08T20:43:28.8558947Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-08T20:43:28.8620149Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-08T20:43:28.8640456Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-08T20:43:28.8700030Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-08T20:43:28.8718238Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-08T20:43:28.8779087Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-08T20:43:28.8799052Z Entering 'third_party/mimalloc' 2024-08-08T20:43:28.8858890Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-08T20:43:28.8878492Z Entering 'third_party/nccl/nccl' 2024-08-08T20:43:28.8937607Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-08T20:43:28.8956893Z Entering 'third_party/nlohmann' 2024-08-08T20:43:28.9016983Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-08T20:43:28.9037060Z Entering 'third_party/onnx' 2024-08-08T20:43:28.9097371Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-08T20:43:28.9134016Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-08T20:43:28.9195430Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-08T20:43:28.9213885Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-08T20:43:28.9274266Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-08T20:43:28.9295377Z Entering 'third_party/opentelemetry-cpp' 2024-08-08T20:43:28.9356651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-08T20:43:28.9377644Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-08T20:43:28.9436704Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-08T20:43:28.9455088Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-08T20:43:28.9514963Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-08T20:43:28.9533033Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-08T20:43:28.9593026Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-08T20:43:28.9611002Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-08T20:43:28.9671811Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-08T20:43:28.9690889Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-08T20:43:28.9750183Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-08T20:43:28.9769739Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-08T20:43:28.9828165Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-08T20:43:28.9846094Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-08T20:43:28.9908482Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-08T20:43:28.9925127Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-08T20:43:28.9985898Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-08T20:43:29.0006494Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-08T20:43:29.0066524Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-08T20:43:29.0086619Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-08T20:43:29.0146500Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-08T20:43:29.0189507Z Entering 'third_party/pocketfft' 2024-08-08T20:43:29.0248058Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-08T20:43:29.0267757Z Entering 'third_party/protobuf' 2024-08-08T20:43:29.0327644Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-08T20:43:29.0349631Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-08T20:43:29.0410801Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-08T20:43:29.0428907Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-08T20:43:29.0492089Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-08T20:43:29.0512039Z Entering 'third_party/psimd' 2024-08-08T20:43:29.0573054Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-08T20:43:29.0591994Z Entering 'third_party/pthreadpool' 2024-08-08T20:43:29.0651970Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-08T20:43:29.0671686Z Entering 'third_party/pybind11' 2024-08-08T20:43:29.0731651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-08T20:43:29.0750880Z Entering 'third_party/python-peachpy' 2024-08-08T20:43:29.0811420Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-08T20:43:29.0830624Z Entering 'third_party/sleef' 2024-08-08T20:43:29.0891453Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-08T20:43:29.0911164Z Entering 'third_party/tensorpipe' 2024-08-08T20:43:29.0972241Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-08T20:43:29.0991182Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-08T20:43:29.1050286Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-08T20:43:29.1068838Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-08T20:43:29.1128492Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-08T20:43:29.1146752Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-08T20:43:29.1205975Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-08T20:43:29.1224899Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-08T20:43:29.1285082Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-08T20:43:29.1302149Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-08T20:43:29.1362078Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-08T20:43:29.2487398Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-08T20:43:29.2839303Z Entering 'android/libs/fbjni' 2024-08-08T20:43:29.2889015Z Entering 'third_party/FP16' 2024-08-08T20:43:29.2938656Z Entering 'third_party/FXdiv' 2024-08-08T20:43:29.2988455Z Entering 'third_party/NNPACK' 2024-08-08T20:43:29.3037455Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-08T20:43:29.3087685Z Entering 'third_party/XNNPACK' 2024-08-08T20:43:29.3154148Z Entering 'third_party/benchmark' 2024-08-08T20:43:29.3203622Z Entering 'third_party/cpp-httplib' 2024-08-08T20:43:29.3253115Z Entering 'third_party/cpuinfo' 2024-08-08T20:43:29.3303235Z Entering 'third_party/cudnn_frontend' 2024-08-08T20:43:29.3352270Z Entering 'third_party/cutlass' 2024-08-08T20:43:29.3411893Z Entering 'third_party/eigen' 2024-08-08T20:43:29.3463961Z Entering 'third_party/fbgemm' 2024-08-08T20:43:29.3513368Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-08T20:43:29.3561365Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-08T20:43:29.3609388Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-08T20:43:29.3664168Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-08T20:43:29.3712301Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-08T20:43:29.3762319Z Entering 'third_party/flatbuffers' 2024-08-08T20:43:29.3814990Z Entering 'third_party/fmt' 2024-08-08T20:43:29.3865838Z Entering 'third_party/foxi' 2024-08-08T20:43:29.3914311Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-08T20:43:29.3965157Z Entering 'third_party/gloo' 2024-08-08T20:43:29.4013716Z Entering 'third_party/googletest' 2024-08-08T20:43:29.4064393Z Entering 'third_party/ideep' 2024-08-08T20:43:29.4112528Z Entering 'third_party/ideep/mkl-dnn' 2024-08-08T20:43:29.4168332Z Entering 'third_party/ittapi' 2024-08-08T20:43:29.4216540Z Entering 'third_party/kineto' 2024-08-08T20:43:29.4267667Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-08T20:43:29.4316151Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-08T20:43:29.4367776Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-08T20:43:29.4415759Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-08T20:43:29.4466037Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-08T20:43:29.4513369Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-08T20:43:29.4565396Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-08T20:43:29.4613434Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-08T20:43:29.4664323Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-08T20:43:29.4714061Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-08T20:43:29.4764844Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-08T20:43:29.4812161Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-08T20:43:29.4862483Z Entering 'third_party/mimalloc' 2024-08-08T20:43:29.4911471Z Entering 'third_party/nccl/nccl' 2024-08-08T20:43:29.4961320Z Entering 'third_party/nlohmann' 2024-08-08T20:43:29.5013350Z Entering 'third_party/onnx' 2024-08-08T20:43:29.5080194Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-08T20:43:29.5128809Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-08T20:43:29.5180060Z Entering 'third_party/opentelemetry-cpp' 2024-08-08T20:43:29.5231359Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-08T20:43:29.5279210Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-08T20:43:29.5327116Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-08T20:43:29.5376799Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-08T20:43:29.5426028Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-08T20:43:29.5474016Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-08T20:43:29.5521820Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-08T20:43:29.5570677Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-08T20:43:29.5621852Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-08T20:43:29.5673896Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-08T20:43:29.5746381Z Entering 'third_party/pocketfft' 2024-08-08T20:43:29.5795611Z Entering 'third_party/protobuf' 2024-08-08T20:43:29.5848888Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-08T20:43:29.5898292Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-08T20:43:29.5948449Z Entering 'third_party/psimd' 2024-08-08T20:43:29.5999697Z Entering 'third_party/pthreadpool' 2024-08-08T20:43:29.6048846Z Entering 'third_party/pybind11' 2024-08-08T20:43:29.6100478Z Entering 'third_party/python-peachpy' 2024-08-08T20:43:29.6149172Z Entering 'third_party/sleef' 2024-08-08T20:43:29.6199092Z Entering 'third_party/tensorpipe' 2024-08-08T20:43:29.6248676Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-08T20:43:29.6298102Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-08T20:43:29.6345922Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-08T20:43:29.6395772Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-08T20:43:29.6443091Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-08T20:43:29.6512511Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-08T20:43:29.6866180Z Entering 'android/libs/fbjni' 2024-08-08T20:43:29.6915206Z Entering 'third_party/FP16' 2024-08-08T20:43:29.6964859Z Entering 'third_party/FXdiv' 2024-08-08T20:43:29.7013732Z Entering 'third_party/NNPACK' 2024-08-08T20:43:29.7063848Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-08T20:43:29.7112528Z Entering 'third_party/XNNPACK' 2024-08-08T20:43:29.7181845Z Entering 'third_party/benchmark' 2024-08-08T20:43:29.7231125Z Entering 'third_party/cpp-httplib' 2024-08-08T20:43:29.7280501Z Entering 'third_party/cpuinfo' 2024-08-08T20:43:29.7329647Z Entering 'third_party/cudnn_frontend' 2024-08-08T20:43:29.7379052Z Entering 'third_party/cutlass' 2024-08-08T20:43:29.7436891Z Entering 'third_party/eigen' 2024-08-08T20:43:29.7489381Z Entering 'third_party/fbgemm' 2024-08-08T20:43:29.7539094Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-08T20:43:29.7588608Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-08T20:43:29.7636562Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-08T20:43:29.7693789Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-08T20:43:29.7742174Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-08T20:43:29.7792822Z Entering 'third_party/flatbuffers' 2024-08-08T20:43:29.7844604Z Entering 'third_party/fmt' 2024-08-08T20:43:29.7893817Z Entering 'third_party/foxi' 2024-08-08T20:43:29.7941750Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-08T20:43:29.7992923Z Entering 'third_party/gloo' 2024-08-08T20:43:29.8041600Z Entering 'third_party/googletest' 2024-08-08T20:43:29.8091424Z Entering 'third_party/ideep' 2024-08-08T20:43:29.8138484Z Entering 'third_party/ideep/mkl-dnn' 2024-08-08T20:43:29.8196089Z Entering 'third_party/ittapi' 2024-08-08T20:43:29.8246216Z Entering 'third_party/kineto' 2024-08-08T20:43:29.8296135Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-08T20:43:29.8344749Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-08T20:43:29.8395512Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-08T20:43:29.8445321Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-08T20:43:29.8495786Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-08T20:43:29.8544406Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-08T20:43:29.8596344Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-08T20:43:29.8644937Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-08T20:43:29.8695300Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-08T20:43:29.8745310Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-08T20:43:29.8796865Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-08T20:43:29.8845249Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-08T20:43:29.8895814Z Entering 'third_party/mimalloc' 2024-08-08T20:43:29.8946172Z Entering 'third_party/nccl/nccl' 2024-08-08T20:43:29.8995891Z Entering 'third_party/nlohmann' 2024-08-08T20:43:29.9045867Z Entering 'third_party/onnx' 2024-08-08T20:43:29.9115269Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-08T20:43:29.9164363Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-08T20:43:29.9215335Z Entering 'third_party/opentelemetry-cpp' 2024-08-08T20:43:29.9267087Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-08T20:43:29.9314182Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-08T20:43:29.9362197Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-08T20:43:29.9409436Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-08T20:43:29.9460572Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-08T20:43:29.9507420Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-08T20:43:29.9555323Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-08T20:43:29.9603709Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-08T20:43:29.9653516Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-08T20:43:29.9705908Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-08T20:43:29.9778859Z Entering 'third_party/pocketfft' 2024-08-08T20:43:29.9826601Z Entering 'third_party/protobuf' 2024-08-08T20:43:29.9879056Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-08T20:43:29.9926860Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-08T20:43:29.9977580Z Entering 'third_party/psimd' 2024-08-08T20:43:30.0026638Z Entering 'third_party/pthreadpool' 2024-08-08T20:43:30.0075793Z Entering 'third_party/pybind11' 2024-08-08T20:43:30.0123913Z Entering 'third_party/python-peachpy' 2024-08-08T20:43:30.0173328Z Entering 'third_party/sleef' 2024-08-08T20:43:30.0222189Z Entering 'third_party/tensorpipe' 2024-08-08T20:43:30.0272764Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-08T20:43:30.0320694Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-08T20:43:30.0369735Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-08T20:43:30.0417754Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-08T20:43:30.0465803Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-08T20:43:30.0531354Z ##[endgroup] 2024-08-08T20:43:30.0579859Z [command]/usr/bin/git log -1 --format='%H' 2024-08-08T20:43:30.0610577Z 'b9d86fa89636e301796d4201f36d86c73f6e49bc' 2024-08-08T20:43:30.0851399Z Prepare all required actions 2024-08-08T20:43:30.0852008Z Getting action download info 2024-08-08T20:43:30.2180540Z ##[group]Run ./.github/actions/setup-linux 2024-08-08T20:43:30.2181097Z env: 2024-08-08T20:43:30.2181429Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:30.2181838Z ##[endgroup] 2024-08-08T20:43:30.2262435Z ##[group]Run set -euo pipefail 2024-08-08T20:43:30.2262981Z set -euo pipefail 2024-08-08T20:43:30.2263448Z function get_ec2_metadata() { 2024-08-08T20:43:30.2264113Z  # Pulled from instance metadata endpoint for EC2 2024-08-08T20:43:30.2265244Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-08T20:43:30.2266211Z  category=$1 2024-08-08T20:43:30.2266867Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-08T20:43:30.2267653Z  runner_name_str=i-09af04530f81e3933 2024-08-08T20:43:30.2268266Z  if [[ -f /.inarc ]]; then 2024-08-08T20:43:30.2268883Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-08T20:43:30.2269564Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-08T20:43:30.2270444Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-08T20:43:30.2271200Z  else 2024-08-08T20:43:30.2271790Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-08T20:43:30.2272502Z  fi 2024-08-08T20:43:30.2272843Z } 2024-08-08T20:43:30.2273260Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-08T20:43:30.2273983Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-08T20:43:30.2274799Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-08T20:43:30.2275489Z echo "system info $(uname -a)" 2024-08-08T20:43:30.2284688Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:30.2285313Z env: 2024-08-08T20:43:30.2285639Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:30.2286073Z ##[endgroup] 2024-08-08T20:43:30.2396846Z ami-id: ami-06c68f701d8090592 2024-08-08T20:43:30.2458375Z instance-id: i-09af04530f81e3933 2024-08-08T20:43:30.2517757Z instance-type: g3.4xlarge 2024-08-08T20:43:30.2530794Z system info Linux ip-10-0-49-49.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-08T20:43:30.2557512Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-08T20:43:30.2559172Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-08T20:43:30.2566326Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:30.2566951Z env: 2024-08-08T20:43:30.2567273Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:30.2567704Z ##[endgroup] 2024-08-08T20:43:30.2649815Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-08T20:43:30.2650572Z if systemctl is-active --quiet docker; then 2024-08-08T20:43:30.2651241Z  echo "Docker daemon is running..."; 2024-08-08T20:43:30.2651994Z else 2024-08-08T20:43:30.2652586Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-08T20:43:30.2653334Z fi 2024-08-08T20:43:30.2660383Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:30.2660998Z env: 2024-08-08T20:43:30.2661334Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:30.2661763Z ##[endgroup] 2024-08-08T20:43:30.2744895Z Docker daemon is running... 2024-08-08T20:43:30.2802593Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-08T20:43:30.2803297Z with: 2024-08-08T20:43:30.2803618Z shell: bash 2024-08-08T20:43:30.2803963Z timeout_minutes: 5 2024-08-08T20:43:30.2804358Z max_attempts: 3 2024-08-08T20:43:30.2804744Z retry_wait_seconds: 30 2024-08-08T20:43:30.2806729Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-08-08T20:43:30.2808668Z polling_interval_seconds: 1 2024-08-08T20:43:30.2809130Z warning_on_retry: true 2024-08-08T20:43:30.2809559Z continue_on_error: false 2024-08-08T20:43:30.2809952Z env: 2024-08-08T20:43:30.2810279Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:30.2839153Z AWS_RETRY_MODE: standard 2024-08-08T20:43:30.2839695Z AWS_MAX_ATTEMPTS: 5 2024-08-08T20:43:30.2840149Z AWS_DEFAULT_REGION: us-east-1 2024-08-08T20:43:30.2840636Z ##[endgroup] 2024-08-08T20:43:31.7501408Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-08T20:43:31.7503403Z Configure a credential helper to remove this warning. See 2024-08-08T20:43:31.7505469Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-08T20:43:31.7506294Z 2024-08-08T20:43:31.7506482Z Login Succeeded 2024-08-08T20:43:32.3588576Z Command completed after 1 attempt(s). 2024-08-08T20:43:32.3648583Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-08T20:43:32.3649501Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-08T20:43:32.3650341Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-08T20:43:32.3658267Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:32.3659566Z env: 2024-08-08T20:43:32.3659917Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:32.3660333Z ##[endgroup] 2024-08-08T20:43:32.3757398Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-08T20:43:32.3758889Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-08T20:43:32.3759594Z # shellcheck disable=SC2046 2024-08-08T20:43:32.3760163Z docker stop $(docker ps -q) || true 2024-08-08T20:43:32.3760752Z # Prune all of the docker images 2024-08-08T20:43:32.3761311Z docker system prune -af 2024-08-08T20:43:32.3768212Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:32.3768852Z env: 2024-08-08T20:43:32.3769192Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:32.3769605Z ##[endgroup] 2024-08-08T20:43:32.4115915Z "docker stop" requires at least 1 argument. 2024-08-08T20:43:32.4116762Z See 'docker stop --help'. 2024-08-08T20:43:32.4117040Z 2024-08-08T20:43:32.4117368Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-08T20:43:32.4117834Z 2024-08-08T20:43:32.4118017Z Stop one or more running containers 2024-08-08T20:43:32.4560847Z Deleted Images: 2024-08-08T20:43:32.4561897Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/tool/alpine:latest 2024-08-08T20:43:32.4563437Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/tool/alpine@sha256:def822f9851ca422481ec6fee59a9966f12b351c62ccb9aca841526ffaa9f748 2024-08-08T20:43:32.4564975Z deleted: sha256:6dbb9cc54074106d46d4ccb330f2a40a682d49dda5f4844962b7dce9fe44aaec 2024-08-08T20:43:32.4566077Z deleted: sha256:b2d5eeeaba3a22b9b8aa97261957974a6bd65274ebd43e1d81d0a7b8b752b116 2024-08-08T20:43:32.4567038Z 2024-08-08T20:43:32.4585375Z Total reclaimed space: 5.613MB 2024-08-08T20:43:32.4624037Z ##[group]Run set +e 2024-08-08T20:43:32.4624485Z set +e 2024-08-08T20:43:32.4624833Z set -x 2024-08-08T20:43:32.4625189Z  2024-08-08T20:43:32.4625584Z PT_DOMAIN=download.pytorch.org 2024-08-08T20:43:32.4626561Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-08T20:43:32.4627928Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-08T20:43:32.4628881Z # one is returned at random 2024-08-08T20:43:32.4629535Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-08T20:43:32.4630170Z  2024-08-08T20:43:32.4630767Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-08T20:43:32.4631681Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-08T20:43:32.4632515Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-08T20:43:32.4633216Z  2024-08-08T20:43:32.4633610Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-08T20:43:32.4634282Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-08T20:43:32.4634917Z  exit 1 2024-08-08T20:43:32.4635305Z  fi 2024-08-08T20:43:32.4635626Z fi 2024-08-08T20:43:32.4635958Z  2024-08-08T20:43:32.4636394Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-08T20:43:32.4637016Z  # Clean up any old records first 2024-08-08T20:43:32.4637648Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-08T20:43:32.4638210Z fi 2024-08-08T20:43:32.4638526Z  2024-08-08T20:43:32.4639064Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-08T20:43:32.4639748Z cat /etc/hosts 2024-08-08T20:43:32.4646812Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:32.4647443Z env: 2024-08-08T20:43:32.4647780Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:32.4648192Z ##[endgroup] 2024-08-08T20:43:32.4679076Z + PT_DOMAIN=download.pytorch.org 2024-08-08T20:43:32.4716130Z ++ dig -4 +short download.pytorch.org 2024-08-08T20:43:32.4716746Z ++ tail -n1 2024-08-08T20:43:32.4927749Z + RESOLVED_IP=18.160.10.28 2024-08-08T20:43:32.4928372Z + '[' -z 18.160.10.28 ']' 2024-08-08T20:43:32.4928897Z + grep -r download.pytorch.org /etc/hosts 2024-08-08T20:43:32.4942193Z 18.160.10.28 download.pytorch.org 2024-08-08T20:43:32.4944351Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-08T20:43:32.6362941Z + echo '18.160.10.28 download.pytorch.org' 2024-08-08T20:43:32.6364292Z + sudo tee -a /etc/hosts 2024-08-08T20:43:32.6912194Z 18.160.10.28 download.pytorch.org 2024-08-08T20:43:32.6931701Z + cat /etc/hosts 2024-08-08T20:43:32.6941832Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-08T20:43:32.6957874Z ::1 localhost6 localhost6.localdomain6 2024-08-08T20:43:32.6959319Z 18.160.10.28 download.pytorch.org 2024-08-08T20:43:32.7166152Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-08T20:43:32.7166949Z with: 2024-08-08T20:43:32.7168172Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:32.7169584Z docker-build-dir: .ci/docker 2024-08-08T20:43:32.7170054Z working-directory: . 2024-08-08T20:43:32.7170624Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:32.7171295Z force-push: false 2024-08-08T20:43:32.7171664Z env: 2024-08-08T20:43:32.7171974Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:32.7172399Z ##[endgroup] 2024-08-08T20:43:32.7196829Z ##[group]Run set -ex 2024-08-08T20:43:32.7197299Z set -ex 2024-08-08T20:43:32.7197663Z  2024-08-08T20:43:32.7198337Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-08T20:43:32.7200687Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-08T20:43:32.7201685Z # job could then download the pre-built image as usual 2024-08-08T20:43:32.7202617Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-08T20:43:32.7203461Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-08T20:43:32.7204254Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-08T20:43:32.7204936Z  2024-08-08T20:43:32.7205574Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-08T20:43:32.7206354Z  exit 0 2024-08-08T20:43:32.7206703Z else 2024-08-08T20:43:32.7207217Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-08T20:43:32.7207788Z fi 2024-08-08T20:43:32.7208103Z  2024-08-08T20:43:32.7208682Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-08T20:43:32.7209780Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-08T20:43:32.7210743Z  # use it as it is, but first let's extract the tag 2024-08-08T20:43:32.7211630Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-08T20:43:32.7212544Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-08T20:43:32.7213418Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-08T20:43:32.7214101Z else 2024-08-08T20:43:32.7214637Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-08T20:43:32.7215468Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-08T20:43:32.7216579Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-08T20:43:32.7217538Z fi 2024-08-08T20:43:32.7225007Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:32.7225629Z env: 2024-08-08T20:43:32.7225960Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:32.7226376Z REPO_NAME: pytorch 2024-08-08T20:43:32.7227659Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:32.7229040Z DOCKER_BUILD_DIR: .ci/docker 2024-08-08T20:43:32.7229655Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:32.7230281Z ##[endgroup] 2024-08-08T20:43:32.7261617Z + [[ ! -d .ci/docker ]] 2024-08-08T20:43:32.7262101Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-08T20:43:32.7262576Z + echo skip=false 2024-08-08T20:43:32.7264678Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-08T20:43:32.7270089Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:32.7271534Z ++ awk -F '[:,]' '{print $2}' 2024-08-08T20:43:32.7295070Z + DOCKER_TAG=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:32.7296117Z + echo docker-tag=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:32.7298175Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:32.7332970Z ##[group]Run set +e 2024-08-08T20:43:32.7333423Z set +e 2024-08-08T20:43:32.7333793Z set -x 2024-08-08T20:43:32.7334127Z  2024-08-08T20:43:32.7334462Z login() { 2024-08-08T20:43:32.7335299Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-08T20:43:32.7336410Z } 2024-08-08T20:43:32.7336732Z  2024-08-08T20:43:32.7337080Z retry () { 2024-08-08T20:43:32.7337548Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-08T20:43:32.7338103Z } 2024-08-08T20:43:32.7338435Z  2024-08-08T20:43:32.7338799Z retry login "${DOCKER_REGISTRY}" 2024-08-08T20:43:32.7339314Z  2024-08-08T20:43:32.7339900Z # Check if image already exists, if it does then skip building it 2024-08-08T20:43:32.7340789Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-08T20:43:32.7341396Z  exit 0 2024-08-08T20:43:32.7341758Z fi 2024-08-08T20:43:32.7342088Z  2024-08-08T20:43:32.7342693Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-08T20:43:32.7343736Z # be empty. The default action would be to continue rebuild the image 2024-08-08T20:43:32.7344667Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-08T20:43:32.7345513Z  # if we're on the base branch then use the parent commit 2024-08-08T20:43:32.7346251Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-08T20:43:32.7346780Z else 2024-08-08T20:43:32.7347345Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-08T20:43:32.7348208Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-08T20:43:32.7348839Z fi 2024-08-08T20:43:32.7349158Z  2024-08-08T20:43:32.7349543Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-08T20:43:32.7350176Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-08T20:43:32.7350741Z  2024-08-08T20:43:32.7351543Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-08T20:43:32.7352500Z  exit 0 2024-08-08T20:43:32.7352861Z fi 2024-08-08T20:43:32.7353174Z  2024-08-08T20:43:32.7353725Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-08T20:43:32.7354962Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-08T20:43:32.7355960Z  exit 1 2024-08-08T20:43:32.7356324Z fi 2024-08-08T20:43:32.7356657Z  2024-08-08T20:43:32.7357244Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-08T20:43:32.7358913Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-08T20:43:32.7360068Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-08T20:43:32.7361298Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-08T20:43:32.7362673Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-08T20:43:32.7363504Z fi 2024-08-08T20:43:32.7363841Z  2024-08-08T20:43:32.7364445Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-08T20:43:32.7371282Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:32.7371912Z env: 2024-08-08T20:43:32.7372233Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:32.7372684Z DOCKER_BUILD_DIR: .ci/docker 2024-08-08T20:43:32.7373267Z BASE_REVISION: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:43:32.7374714Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:32.7376178Z DOCKER_TAG: 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:32.7376950Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:32.7377591Z ##[endgroup] 2024-08-08T20:43:32.7406431Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:32.7407303Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:32.7409588Z + aws ecr get-login-password --region us-east-1 2024-08-08T20:43:32.7411979Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:33.4430494Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-08T20:43:33.4431846Z Configure a credential helper to remove this warning. See 2024-08-08T20:43:33.4433013Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-08T20:43:33.4433686Z 2024-08-08T20:43:33.4433836Z Login Succeeded 2024-08-08T20:43:33.4445585Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:33.7129754Z { 2024-08-08T20:43:33.7130452Z "schemaVersion": 2, 2024-08-08T20:43:33.7131301Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-08T20:43:33.7132030Z "config": { 2024-08-08T20:43:33.7132669Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-08T20:43:33.7133326Z "size": 49379, 2024-08-08T20:43:33.7134012Z "digest": "sha256:cd235e6b1be7a0d3099ad6f7c05391c21669cd859ddf645fc846603ac8b56d9d" 2024-08-08T20:43:33.7134794Z }, 2024-08-08T20:43:33.7135766Z "layers": [ 2024-08-08T20:43:33.7136369Z { 2024-08-08T20:43:33.7137452Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7138570Z "size": 28584317, 2024-08-08T20:43:33.7139222Z "digest": "sha256:63e9bbe323274e77e58d77c6ab6802d247458f784222fbb07a2556d6ec74ee05" 2024-08-08T20:43:33.7139999Z }, 2024-08-08T20:43:33.7140276Z { 2024-08-08T20:43:33.7140818Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7141514Z "size": 7944771, 2024-08-08T20:43:33.7142193Z "digest": "sha256:cfb3d849840ee60cee7b02bad68c1fc3c15928ebcd88f327754766b670578ed6" 2024-08-08T20:43:33.7142983Z }, 2024-08-08T20:43:33.7143279Z { 2024-08-08T20:43:33.7143821Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7144517Z "size": 57593718, 2024-08-08T20:43:33.7145188Z "digest": "sha256:968831e596a6288f0fed9b8612ee4ee8e75511037c4305058805492c5162e481" 2024-08-08T20:43:33.7145927Z }, 2024-08-08T20:43:33.7146220Z { 2024-08-08T20:43:33.7146748Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7147429Z "size": 187, 2024-08-08T20:43:33.7148109Z "digest": "sha256:ea310eb267ca1cab61b6b16f566cd28bfd59a741395a011f5e76716e15ba57c6" 2024-08-08T20:43:33.7148895Z }, 2024-08-08T20:43:33.7149177Z { 2024-08-08T20:43:33.7149708Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7150400Z "size": 6885, 2024-08-08T20:43:33.7151064Z "digest": "sha256:3af11d09e9cd1eb9c379f0a4071231e5a5642eb728b4b33bcb76be291f3c9488" 2024-08-08T20:43:33.7151848Z }, 2024-08-08T20:43:33.7152142Z { 2024-08-08T20:43:33.7152981Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7154964Z "size": 1361380219, 2024-08-08T20:43:33.7155824Z "digest": "sha256:ebfec18059b91e56882881ac34754f917861edb5f732c395d2a1a851bbd6db46" 2024-08-08T20:43:33.7156517Z }, 2024-08-08T20:43:33.7156815Z { 2024-08-08T20:43:33.7157350Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7158027Z "size": 62686, 2024-08-08T20:43:33.7159322Z "digest": "sha256:533b4aebf16914c763b7b0de3ce657590c6f979045e9fdf1f816adaf68d8a4d3" 2024-08-08T20:43:33.7160127Z }, 2024-08-08T20:43:33.7160403Z { 2024-08-08T20:43:33.7160936Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7161634Z "size": 1685, 2024-08-08T20:43:33.7162287Z "digest": "sha256:9dd75d06a0910f28cb1e484b8808724e5a6ee570ecb8fc04631368f546b39ed9" 2024-08-08T20:43:33.7163060Z }, 2024-08-08T20:43:33.7163360Z { 2024-08-08T20:43:33.7163878Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7164811Z "size": 1523, 2024-08-08T20:43:33.7165498Z "digest": "sha256:30bfca4dd3492d60ed8035b0eeb1229897041140db117f5663465a551e25851d" 2024-08-08T20:43:33.7166278Z }, 2024-08-08T20:43:33.7166559Z { 2024-08-08T20:43:33.7167101Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7167785Z "size": 2608186849, 2024-08-08T20:43:33.7168502Z "digest": "sha256:1b57ce94cad9a5097d99b8d6f7bcd51df5a162fc3c5f5686b689b76993724bc8" 2024-08-08T20:43:33.7169287Z }, 2024-08-08T20:43:33.7169569Z { 2024-08-08T20:43:33.7170105Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7170805Z "size": 86631, 2024-08-08T20:43:33.7171492Z "digest": "sha256:9ee6bdb31195dd42fd98147a75d540efe0c5708e0ecda866a0bca060084fddab" 2024-08-08T20:43:33.7172256Z }, 2024-08-08T20:43:33.7172551Z { 2024-08-08T20:43:33.7173072Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7173778Z "size": 1823, 2024-08-08T20:43:33.7174478Z "digest": "sha256:08be3f7f3cfb106016f7b52ce7babbc0fff221bbc8422880fd11f8045fb04e39" 2024-08-08T20:43:33.7175269Z }, 2024-08-08T20:43:33.7175550Z { 2024-08-08T20:43:33.7176085Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7176777Z "size": 245786667, 2024-08-08T20:43:33.7177465Z "digest": "sha256:fd57b6413183c2eafa95d909f3ee48a6d30511178a96333ee4923bb9edc28aa5" 2024-08-08T20:43:33.7178237Z }, 2024-08-08T20:43:33.7178532Z { 2024-08-08T20:43:33.7179050Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7179739Z "size": 545, 2024-08-08T20:43:33.7180425Z "digest": "sha256:9a1bc15482bfb03e613a31fef4f75591cdcd6158faabd8bbf3ddf3f37439737d" 2024-08-08T20:43:33.7181200Z }, 2024-08-08T20:43:33.7181500Z { 2024-08-08T20:43:33.7182034Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7182707Z "size": 1284, 2024-08-08T20:43:33.7183404Z "digest": "sha256:909b1bf15eccb3fd953943989fa974c79d52c18ac03cb592daa71daa0dc415cc" 2024-08-08T20:43:33.7184186Z }, 2024-08-08T20:43:33.7184465Z { 2024-08-08T20:43:33.7184998Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7185690Z "size": 484, 2024-08-08T20:43:33.7186347Z "digest": "sha256:88e7a278ede8a729ae3657c3c7a836a5eeb3229c5c36ab6723cd030b9e7abe85" 2024-08-08T20:43:33.7187122Z }, 2024-08-08T20:43:33.7187419Z { 2024-08-08T20:43:33.7187930Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7188620Z "size": 91713067, 2024-08-08T20:43:33.7189307Z "digest": "sha256:e5d7856b663f150b51762230d693bb44ae2bcbb09a40431bc6dcbf73f9f3e871" 2024-08-08T20:43:33.7190058Z }, 2024-08-08T20:43:33.7190354Z { 2024-08-08T20:43:33.7190887Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7191561Z "size": 3229, 2024-08-08T20:43:33.7192242Z "digest": "sha256:00a0e1fafdad8e9d907d3fc7f0231a49e06158f37329ebf99bc2fd47c70866df" 2024-08-08T20:43:33.7193180Z }, 2024-08-08T20:43:33.7193487Z { 2024-08-08T20:43:33.7194020Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7194712Z "size": 1909, 2024-08-08T20:43:33.7195373Z "digest": "sha256:b137e3b1350c038bcf530e9081f56d6b79a94cb2c8c56da7aebd04e69ec09c77" 2024-08-08T20:43:33.7196150Z }, 2024-08-08T20:43:33.7196445Z { 2024-08-08T20:43:33.7196963Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7197657Z "size": 700, 2024-08-08T20:43:33.7198326Z "digest": "sha256:fc51810b1926e7228fbf5d689aae1d1dd5a5d720d28318406725a284fd6ec07c" 2024-08-08T20:43:33.7199086Z }, 2024-08-08T20:43:33.7199384Z { 2024-08-08T20:43:33.7199919Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7200597Z "size": 2801669971, 2024-08-08T20:43:33.7201316Z "digest": "sha256:fbcb08d9f4bc971495bd9eeea6116fb1efd8e1b13008d82139c326d75c07a7d5" 2024-08-08T20:43:33.7202226Z }, 2024-08-08T20:43:33.7202515Z { 2024-08-08T20:43:33.7203050Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7203739Z "size": 381, 2024-08-08T20:43:33.7204385Z "digest": "sha256:4192f5853f54248539aefdbe4921d763d158bcde68eabe869e412111d8e29739" 2024-08-08T20:43:33.7205153Z }, 2024-08-08T20:43:33.7205446Z { 2024-08-08T20:43:33.7205960Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7206652Z "size": 12893, 2024-08-08T20:43:33.7207378Z "digest": "sha256:62e5e179e864ff5209fd3a8fc2022df745a07048b3142aea9d90450b85a79f46" 2024-08-08T20:43:33.7208125Z }, 2024-08-08T20:43:33.7208420Z { 2024-08-08T20:43:33.7208950Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7209619Z "size": 803, 2024-08-08T20:43:33.7210282Z "digest": "sha256:3e326a36ef0f1bb7289f721af48968221079ef67dccb0130fa29f6d1e2ffffaa" 2024-08-08T20:43:33.7211068Z }, 2024-08-08T20:43:33.7211379Z { 2024-08-08T20:43:33.7211906Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7212603Z "size": 106, 2024-08-08T20:43:33.7213266Z "digest": "sha256:56d2d1e08c75d74d55ce83ba1f530347659846b032da1f0389ec10ef0e895943" 2024-08-08T20:43:33.7214018Z }, 2024-08-08T20:43:33.7214315Z { 2024-08-08T20:43:33.7214848Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7215519Z "size": 504, 2024-08-08T20:43:33.7216173Z "digest": "sha256:438d85947b431d78959c32a400342dc72d5b95935ce3f0640fb21aa21df254d0" 2024-08-08T20:43:33.7216932Z }, 2024-08-08T20:43:33.7217211Z { 2024-08-08T20:43:33.7217747Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7218441Z "size": 121477346, 2024-08-08T20:43:33.7219130Z "digest": "sha256:65072a4267b4e36b3af50ebd6d8bb81906d21b5320d11a00221adbb8b6495c05" 2024-08-08T20:43:33.7219909Z }, 2024-08-08T20:43:33.7220215Z { 2024-08-08T20:43:33.7220738Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7221432Z "size": 109, 2024-08-08T20:43:33.7222115Z "digest": "sha256:c88cf310204cc37c4bcffa5711ff941f931dfa5b05bcb7728c89ac6967b9b764" 2024-08-08T20:43:33.7222899Z }, 2024-08-08T20:43:33.7223179Z { 2024-08-08T20:43:33.7223709Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7224457Z "size": 489, 2024-08-08T20:43:33.7225076Z "digest": "sha256:a54e8a071aefefb71af4568e1db1da2d7f95f83c11334bf5da54c990dd5d9a60" 2024-08-08T20:43:33.7225871Z }, 2024-08-08T20:43:33.7226168Z { 2024-08-08T20:43:33.7226685Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7227377Z "size": 297, 2024-08-08T20:43:33.7228051Z "digest": "sha256:37cedc534d8875f5e6a116df8843321b6f12d2baef68d7e38a8f784c69e17522" 2024-08-08T20:43:33.7228815Z }, 2024-08-08T20:43:33.7229111Z { 2024-08-08T20:43:33.7229644Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7230437Z "size": 103, 2024-08-08T20:43:33.7231140Z "digest": "sha256:33a4aacd6d0ee8d26a8f108968c61b30bbde076c8412b495b27fdd1376accf3f" 2024-08-08T20:43:33.7231932Z }, 2024-08-08T20:43:33.7232213Z { 2024-08-08T20:43:33.7232750Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7233447Z "size": 1473, 2024-08-08T20:43:33.7234095Z "digest": "sha256:c5585171af7c07862130070fcab84d7ed9f31e6980420fb38f443bceaf612884" 2024-08-08T20:43:33.7234863Z }, 2024-08-08T20:43:33.7235222Z { 2024-08-08T20:43:33.7235678Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7236373Z "size": 424146559, 2024-08-08T20:43:33.7237087Z "digest": "sha256:f956a95cc728a3aee9c3c8bc0b32ebfe598f92dc4cc11ee88fcc17f5cd4bce29" 2024-08-08T20:43:33.7237866Z }, 2024-08-08T20:43:33.7238161Z { 2024-08-08T20:43:33.7238691Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7239496Z "size": 160, 2024-08-08T20:43:33.7240181Z "digest": "sha256:57f752d95ca739b39ff7b9fbc615535f3964a9f8dff4aebc5b050f061f2da576" 2024-08-08T20:43:33.7240965Z }, 2024-08-08T20:43:33.7241243Z { 2024-08-08T20:43:33.7241776Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7242470Z "size": 563, 2024-08-08T20:43:33.7243105Z "digest": "sha256:529bfa1f03893c15b190023497229174fb0a85f98ef370614f33fd6939524704" 2024-08-08T20:43:33.7243859Z }, 2024-08-08T20:43:33.7244155Z { 2024-08-08T20:43:33.7244672Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7245369Z "size": 35867002, 2024-08-08T20:43:33.7246062Z "digest": "sha256:241d45dc1ded76b86b1c877d5887f1692cbf77a7c32b11d5118284993f679510" 2024-08-08T20:43:33.7246816Z }, 2024-08-08T20:43:33.7247114Z { 2024-08-08T20:43:33.7247650Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7248337Z "size": 104, 2024-08-08T20:43:33.7249022Z "digest": "sha256:01e679af7dbce49ea38c30a7f4ed07e5eb6d4986b2ca65458e39d8f1d178a6fc" 2024-08-08T20:43:33.7249806Z }, 2024-08-08T20:43:33.7250086Z { 2024-08-08T20:43:33.7250616Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7251308Z "size": 425, 2024-08-08T20:43:33.7251965Z "digest": "sha256:667d9aa101b1e662567dbbd3e5e9536c9829754dc5ba9dc5dfc57a4c1dfbfaad" 2024-08-08T20:43:33.7252742Z }, 2024-08-08T20:43:33.7253041Z { 2024-08-08T20:43:33.7253555Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7254246Z "size": 20262086, 2024-08-08T20:43:33.7254947Z "digest": "sha256:4597a1cd71c8273d55d59fd733edbe8b77dfa46fa62424313aa610e00bcee9db" 2024-08-08T20:43:33.7255698Z }, 2024-08-08T20:43:33.7255994Z { 2024-08-08T20:43:33.7256526Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7257201Z "size": 439, 2024-08-08T20:43:33.7257893Z "digest": "sha256:d5d1806bb65ac40856eb41fed88289f4565d3c2eeccbc7f13424e91e6515bbd1" 2024-08-08T20:43:33.7259140Z }, 2024-08-08T20:43:33.7259428Z { 2024-08-08T20:43:33.7259963Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7260653Z "size": 700, 2024-08-08T20:43:33.7261304Z "digest": "sha256:fc51810b1926e7228fbf5d689aae1d1dd5a5d720d28318406725a284fd6ec07c" 2024-08-08T20:43:33.7262075Z }, 2024-08-08T20:43:33.7262375Z { 2024-08-08T20:43:33.7262887Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7263577Z "size": 142, 2024-08-08T20:43:33.7264238Z "digest": "sha256:2090aa744d9926715bd4f4e71e7275d460c24ddee297c235e133ae52f6d6557b" 2024-08-08T20:43:33.7264990Z }, 2024-08-08T20:43:33.7265287Z { 2024-08-08T20:43:33.7265819Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7266496Z "size": 135, 2024-08-08T20:43:33.7267175Z "digest": "sha256:508b81f2bbecf87861845ddd9affe1e1ce7717d82c2d8d8446c94d0ae7bb634c" 2024-08-08T20:43:33.7268154Z }, 2024-08-08T20:43:33.7268461Z { 2024-08-08T20:43:33.7268998Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7269691Z "size": 32, 2024-08-08T20:43:33.7270354Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-08T20:43:33.7271133Z }, 2024-08-08T20:43:33.7271431Z { 2024-08-08T20:43:33.7271942Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7272631Z "size": 190, 2024-08-08T20:43:33.7273305Z "digest": "sha256:6501ca0d0154cc806fe5d4b0b62dd865fcbf9981be3236045a2d23ced95b3662" 2024-08-08T20:43:33.7274063Z }, 2024-08-08T20:43:33.7274356Z { 2024-08-08T20:43:33.7274888Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7275564Z "size": 565, 2024-08-08T20:43:33.7276238Z "digest": "sha256:56781236accc9f35d8bba6ab5c965b782e312713eed0b11ab6a623caf5e7c7dc" 2024-08-08T20:43:33.7277165Z }, 2024-08-08T20:43:33.7277443Z { 2024-08-08T20:43:33.7277986Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7278683Z "size": 43163828, 2024-08-08T20:43:33.7279347Z "digest": "sha256:3794293299cd9060f62bdfd80afba731e6643595c0454f57e8635c11299d3970" 2024-08-08T20:43:33.7280111Z }, 2024-08-08T20:43:33.7280408Z { 2024-08-08T20:43:33.7280919Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7281612Z "size": 106, 2024-08-08T20:43:33.7282292Z "digest": "sha256:a4bf37ae774bb6fb81ef08ad416a3f326560824c22261e3af8128e4d1215ebea" 2024-08-08T20:43:33.7283049Z }, 2024-08-08T20:43:33.7283347Z { 2024-08-08T20:43:33.7283884Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7284555Z "size": 1212, 2024-08-08T20:43:33.7285237Z "digest": "sha256:5f92aea02160822d753bb8f832aa0cabeb20fccbb172c708245de3a3b41bbadc" 2024-08-08T20:43:33.7286018Z }, 2024-08-08T20:43:33.7286303Z { 2024-08-08T20:43:33.7286842Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7287538Z "size": 700, 2024-08-08T20:43:33.7288191Z "digest": "sha256:fc51810b1926e7228fbf5d689aae1d1dd5a5d720d28318406725a284fd6ec07c" 2024-08-08T20:43:33.7288966Z }, 2024-08-08T20:43:33.7289274Z { 2024-08-08T20:43:33.7289792Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7290483Z "size": 137, 2024-08-08T20:43:33.7291152Z "digest": "sha256:3976563a126eae89f7e0b04ae1298bb24537eb8dd20d527857d59affeeef0a69" 2024-08-08T20:43:33.7291905Z }, 2024-08-08T20:43:33.7292201Z { 2024-08-08T20:43:33.7292735Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7293402Z "size": 120, 2024-08-08T20:43:33.7294050Z "digest": "sha256:1457a83f811084f2f2e97af9a90728d3816379fc8966bd6a4f13571f15c5411b" 2024-08-08T20:43:33.7294805Z }, 2024-08-08T20:43:33.7295083Z { 2024-08-08T20:43:33.7295625Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7296327Z "size": 1916631161, 2024-08-08T20:43:33.7297013Z "digest": "sha256:f4957e5566dfa8c923498bbccf987233d635fae2478a7ad7dc1c56fa068566b4" 2024-08-08T20:43:33.7297784Z }, 2024-08-08T20:43:33.7298154Z { 2024-08-08T20:43:33.7298598Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7299286Z "size": 172, 2024-08-08T20:43:33.7299984Z "digest": "sha256:ea28e901cc2b64d1805adae2e3de9efae7ce5b497c0b5ae30d6dda7286e85b59" 2024-08-08T20:43:33.7300761Z }, 2024-08-08T20:43:33.7301056Z { 2024-08-08T20:43:33.7301584Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7302255Z "size": 907, 2024-08-08T20:43:33.7302914Z "digest": "sha256:128f963e326f8113050faaf9dc9dce161b567b56ef9b3359eef6948968bc6ba5" 2024-08-08T20:43:33.7303684Z }, 2024-08-08T20:43:33.7303960Z { 2024-08-08T20:43:33.7304485Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7305289Z "size": 700, 2024-08-08T20:43:33.7305965Z "digest": "sha256:fc51810b1926e7228fbf5d689aae1d1dd5a5d720d28318406725a284fd6ec07c" 2024-08-08T20:43:33.7306734Z }, 2024-08-08T20:43:33.7307077Z { 2024-08-08T20:43:33.7307591Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7308274Z "size": 134, 2024-08-08T20:43:33.7309002Z "digest": "sha256:0e037b263d65bc1bbf9b2259030d48c11336cfe1e7003c800b00b761c26428f6" 2024-08-08T20:43:33.7309690Z }, 2024-08-08T20:43:33.7309983Z { 2024-08-08T20:43:33.7310521Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7311192Z "size": 32, 2024-08-08T20:43:33.7311861Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-08T20:43:33.7312645Z }, 2024-08-08T20:43:33.7312924Z { 2024-08-08T20:43:33.7313457Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7314265Z "size": 156, 2024-08-08T20:43:33.7314900Z "digest": "sha256:75143e4e0c7767a475016c209b46830984e3f7389a25ff4a6d9876685802c057" 2024-08-08T20:43:33.7315646Z }, 2024-08-08T20:43:33.7315944Z { 2024-08-08T20:43:33.7316459Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7317148Z "size": 1841, 2024-08-08T20:43:33.7317825Z "digest": "sha256:5f2762373cead6b01747aff43a5fca693608d689e4c0ab8f27f592d2c06c19fc" 2024-08-08T20:43:33.7318581Z }, 2024-08-08T20:43:33.7318877Z { 2024-08-08T20:43:33.7319406Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7320080Z "size": 7529784, 2024-08-08T20:43:33.7320765Z "digest": "sha256:b0fdf0dc9c7d4594c8bcb1938d0554a76479140d120b539fd09d8a79c0fa3062" 2024-08-08T20:43:33.7321537Z }, 2024-08-08T20:43:33.7321812Z { 2024-08-08T20:43:33.7322341Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7323031Z "size": 164, 2024-08-08T20:43:33.7323720Z "digest": "sha256:7cf5bf391d8b5fbe7c63b34e2bbd993b6eb25d5662376f16e6ca06bc1d6eb6c9" 2024-08-08T20:43:33.7324509Z }, 2024-08-08T20:43:33.7324804Z { 2024-08-08T20:43:33.7325317Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7326012Z "size": 7944, 2024-08-08T20:43:33.7326703Z "digest": "sha256:c274269f5d7e256bbe90ebbc9b5218d6aafffa5b2e3abea03c095f59eaf3ee27" 2024-08-08T20:43:33.7327476Z }, 2024-08-08T20:43:33.7327769Z { 2024-08-08T20:43:33.7328300Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7328976Z "size": 8069, 2024-08-08T20:43:33.7329660Z "digest": "sha256:163b636fa1af9a323822eff8881a3fceb1fd8da7c633ce060895c3a4aa93f594" 2024-08-08T20:43:33.7330442Z }, 2024-08-08T20:43:33.7330721Z { 2024-08-08T20:43:33.7331250Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7331943Z "size": 301, 2024-08-08T20:43:33.7332588Z "digest": "sha256:21c5c3d188ac93a5846c6147ef3b23001e97ce8dc9765f5e236710ee84b127f8" 2024-08-08T20:43:33.7333379Z }, 2024-08-08T20:43:33.7333675Z { 2024-08-08T20:43:33.7334189Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7334879Z "size": 7630427, 2024-08-08T20:43:33.7335554Z "digest": "sha256:8befc1fd9685802235434744164c49e50cd83403b4de685426ba7f7fd9dfeaa5" 2024-08-08T20:43:33.7336295Z }, 2024-08-08T20:43:33.7336592Z { 2024-08-08T20:43:33.7337123Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7337799Z "size": 108, 2024-08-08T20:43:33.7338462Z "digest": "sha256:6061510659f35eaeafca4b18e7d6769ff2d0793bd5a7c524b0d853c183131830" 2024-08-08T20:43:33.7339235Z }, 2024-08-08T20:43:33.7371781Z { 2024-08-08T20:43:33.7372415Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7373135Z "size": 54145774, 2024-08-08T20:43:33.7373851Z "digest": "sha256:1baeb6c7bd597982ba5c766b7723957e2c53a616da85f1ea79a0bbbefeaa1bb8" 2024-08-08T20:43:33.7374643Z }, 2024-08-08T20:43:33.7375149Z { 2024-08-08T20:43:33.7375722Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7376408Z "size": 473, 2024-08-08T20:43:33.7377091Z "digest": "sha256:8b31ac4f8898b741c2b9f5ded6e82541cd590de3f0682d37cfdc828d9162674b" 2024-08-08T20:43:33.7377875Z }, 2024-08-08T20:43:33.7378156Z { 2024-08-08T20:43:33.7378696Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7379387Z "size": 1374858366, 2024-08-08T20:43:33.7380080Z "digest": "sha256:64f3015ce9ddf56cae992548f3a7d61fee39a0c9fca4aa9c2aeb66a6413c8877" 2024-08-08T20:43:33.7380875Z }, 2024-08-08T20:43:33.7381165Z { 2024-08-08T20:43:33.7381678Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7382371Z "size": 106, 2024-08-08T20:43:33.7383030Z "digest": "sha256:ee76d9405e95828d1c7f1859f3b9f4f755a9521b24fee3eca27650a3acef9923" 2024-08-08T20:43:33.7383945Z }, 2024-08-08T20:43:33.7384245Z { 2024-08-08T20:43:33.7384773Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7385447Z "size": 558, 2024-08-08T20:43:33.7386123Z "digest": "sha256:35b0f99cd85f08c7dead3225af75e654daaf96653d7516e8a3dcb0f8ad383803" 2024-08-08T20:43:33.7386893Z }, 2024-08-08T20:43:33.7387171Z { 2024-08-08T20:43:33.7387706Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7388385Z "size": 46248419, 2024-08-08T20:43:33.7389023Z "digest": "sha256:cd4355139790949344d68191a7fb9f145f75eb88b58b071e855aadb7e9f68897" 2024-08-08T20:43:33.7389750Z }, 2024-08-08T20:43:33.7390029Z { 2024-08-08T20:43:33.7390527Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7391189Z "size": 111, 2024-08-08T20:43:33.7391851Z "digest": "sha256:6520dd9eea8aa7f640cc677ab3057a3cbad7331a8dc5e597a5038425b9f77b33" 2024-08-08T20:43:33.7392619Z }, 2024-08-08T20:43:33.7392933Z { 2024-08-08T20:43:33.7393472Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7394150Z "size": 32, 2024-08-08T20:43:33.7394825Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-08T20:43:33.7395610Z }, 2024-08-08T20:43:33.7395892Z { 2024-08-08T20:43:33.7396454Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7397143Z "size": 32, 2024-08-08T20:43:33.7397792Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-08T20:43:33.7398568Z }, 2024-08-08T20:43:33.7398863Z { 2024-08-08T20:43:33.7399374Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7400068Z "size": 32, 2024-08-08T20:43:33.7400732Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-08T20:43:33.7401490Z }, 2024-08-08T20:43:33.7401770Z { 2024-08-08T20:43:33.7402282Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-08T20:43:33.7402938Z "size": 32, 2024-08-08T20:43:33.7403578Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-08T20:43:33.7404337Z } 2024-08-08T20:43:33.7404616Z ] 2024-08-08T20:43:33.7404899Z } 2024-08-08T20:43:33.7405217Z + exit 0 2024-08-08T20:43:33.7538532Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-08T20:43:33.7539169Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-08T20:43:33.7539842Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-08T20:43:33.7547208Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:33.7547830Z env: 2024-08-08T20:43:33.7548169Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:33.7549474Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:33.7550807Z ##[endgroup] 2024-08-08T20:43:33.7584263Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:33.7647266Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-08T20:43:33.7648029Z with: 2024-08-08T20:43:33.7649223Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:33.7650765Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:33.7651393Z env: 2024-08-08T20:43:33.7651720Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:33.7652147Z ##[endgroup] 2024-08-08T20:43:33.7674097Z ##[group]Run set -x 2024-08-08T20:43:33.7674518Z set -x 2024-08-08T20:43:33.7674882Z set +e 2024-08-08T20:43:33.7675236Z  2024-08-08T20:43:33.7675552Z login() { 2024-08-08T20:43:33.7676380Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-08T20:43:33.7677488Z } 2024-08-08T20:43:33.7677802Z  2024-08-08T20:43:33.7678198Z retry () { 2024-08-08T20:43:33.7678664Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-08T20:43:33.7679216Z } 2024-08-08T20:43:33.7679546Z  2024-08-08T20:43:33.7679910Z retry login "${DOCKER_REGISTRY}" 2024-08-08T20:43:33.7680421Z  2024-08-08T20:43:33.7680750Z set -e 2024-08-08T20:43:33.7681317Z # ignore output since only exit code is used for conditional 2024-08-08T20:43:33.7682198Z # only pull docker image if it's not available locally 2024-08-08T20:43:33.7683150Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-08T20:43:33.7684019Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-08T20:43:33.7684543Z fi 2024-08-08T20:43:33.7690912Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:43:33.7691553Z env: 2024-08-08T20:43:33.7691872Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:43:33.7693168Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:33.7694676Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:33.7695305Z ##[endgroup] 2024-08-08T20:43:33.7723090Z + set +e 2024-08-08T20:43:33.7723952Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:33.7724748Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:33.7727394Z + aws ecr get-login-password --region us-east-1 2024-08-08T20:43:33.7728879Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-08T20:43:34.4651860Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-08T20:43:34.4652875Z Configure a credential helper to remove this warning. See 2024-08-08T20:43:34.4654243Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-08T20:43:34.4654928Z 2024-08-08T20:43:34.4655117Z Login Succeeded 2024-08-08T20:43:34.4666484Z + set -e 2024-08-08T20:43:34.4668039Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:34.4837295Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:34.4839686Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:43:34.7426407Z 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9: Pulling from pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9 2024-08-08T20:43:34.7428003Z 63e9bbe32327: Pulling fs layer 2024-08-08T20:43:34.7428714Z cfb3d849840e: Pulling fs layer 2024-08-08T20:43:34.7429200Z 968831e596a6: Pulling fs layer 2024-08-08T20:43:34.7429992Z ea310eb267ca: Pulling fs layer 2024-08-08T20:43:34.7430837Z 3af11d09e9cd: Pulling fs layer 2024-08-08T20:43:34.7431711Z ebfec18059b9: Pulling fs layer 2024-08-08T20:43:34.7432623Z 533b4aebf169: Pulling fs layer 2024-08-08T20:43:34.7433528Z 9dd75d06a091: Pulling fs layer 2024-08-08T20:43:34.7434449Z 30bfca4dd349: Pulling fs layer 2024-08-08T20:43:34.7434985Z 1b57ce94cad9: Pulling fs layer 2024-08-08T20:43:34.7435435Z 9ee6bdb31195: Pulling fs layer 2024-08-08T20:43:34.7436280Z 08be3f7f3cfb: Pulling fs layer 2024-08-08T20:43:34.7436783Z fd57b6413183: Pulling fs layer 2024-08-08T20:43:34.7437655Z 9a1bc15482bf: Pulling fs layer 2024-08-08T20:43:34.7438408Z 909b1bf15ecc: Pulling fs layer 2024-08-08T20:43:34.7439262Z 88e7a278ede8: Pulling fs layer 2024-08-08T20:43:34.7439954Z e5d7856b663f: Pulling fs layer 2024-08-08T20:43:34.7440372Z 533b4aebf169: Waiting 2024-08-08T20:43:34.7440880Z 00a0e1fafdad: Pulling fs layer 2024-08-08T20:43:34.7441872Z b137e3b1350c: Pulling fs layer 2024-08-08T20:43:34.7442288Z 9dd75d06a091: Waiting 2024-08-08T20:43:34.7442729Z fc51810b1926: Pulling fs layer 2024-08-08T20:43:34.7443181Z 30bfca4dd349: Waiting 2024-08-08T20:43:34.7443539Z fbcb08d9f4bc: Pulling fs layer 2024-08-08T20:43:34.7443935Z ea310eb267ca: Waiting 2024-08-08T20:43:34.7444327Z 1b57ce94cad9: Waiting 2024-08-08T20:43:34.7444766Z 4192f5853f54: Pulling fs layer 2024-08-08T20:43:34.7445195Z 62e5e179e864: Pulling fs layer 2024-08-08T20:43:34.7445705Z 9ee6bdb31195: Waiting 2024-08-08T20:43:34.7445987Z 3af11d09e9cd: Waiting 2024-08-08T20:43:34.7446369Z 9a1bc15482bf: Waiting 2024-08-08T20:43:34.7446755Z 08be3f7f3cfb: Waiting 2024-08-08T20:43:34.7447126Z 909b1bf15ecc: Waiting 2024-08-08T20:43:34.7447563Z 3e326a36ef0f: Pulling fs layer 2024-08-08T20:43:34.7448175Z fd57b6413183: Waiting 2024-08-08T20:43:34.7448873Z 56d2d1e08c75: Pulling fs layer 2024-08-08T20:43:34.7449398Z fc51810b1926: Waiting 2024-08-08T20:43:34.7449752Z 88e7a278ede8: Waiting 2024-08-08T20:43:34.7450237Z ebfec18059b9: Waiting 2024-08-08T20:43:34.7450645Z e5d7856b663f: Waiting 2024-08-08T20:43:34.7451020Z 438d85947b43: Pulling fs layer 2024-08-08T20:43:34.7451542Z b137e3b1350c: Waiting 2024-08-08T20:43:34.7451851Z 00a0e1fafdad: Waiting 2024-08-08T20:43:34.7452238Z 65072a4267b4: Pulling fs layer 2024-08-08T20:43:34.7452677Z 3e326a36ef0f: Waiting 2024-08-08T20:43:34.7453064Z fbcb08d9f4bc: Waiting 2024-08-08T20:43:34.7453450Z c88cf310204c: Pulling fs layer 2024-08-08T20:43:34.7453890Z 56d2d1e08c75: Waiting 2024-08-08T20:43:34.7454270Z 4192f5853f54: Waiting 2024-08-08T20:43:34.7454652Z a54e8a071aef: Pulling fs layer 2024-08-08T20:43:34.7455090Z 438d85947b43: Waiting 2024-08-08T20:43:34.7455463Z 65072a4267b4: Waiting 2024-08-08T20:43:34.7455846Z 37cedc534d88: Pulling fs layer 2024-08-08T20:43:34.7456284Z c88cf310204c: Waiting 2024-08-08T20:43:34.7456666Z a54e8a071aef: Waiting 2024-08-08T20:43:34.7457118Z 33a4aacd6d0e: Pulling fs layer 2024-08-08T20:43:34.7457587Z 37cedc534d88: Waiting 2024-08-08T20:43:34.7457910Z c5585171af7c: Pulling fs layer 2024-08-08T20:43:34.7458826Z f956a95cc728: Pulling fs layer 2024-08-08T20:43:34.7459276Z 57f752d95ca7: Pulling fs layer 2024-08-08T20:43:34.7459732Z 529bfa1f0389: Pulling fs layer 2024-08-08T20:43:34.7460150Z 57f752d95ca7: Waiting 2024-08-08T20:43:34.7460536Z f956a95cc728: Waiting 2024-08-08T20:43:34.7460920Z 33a4aacd6d0e: Waiting 2024-08-08T20:43:34.7461305Z 241d45dc1ded: Pulling fs layer 2024-08-08T20:43:34.7461769Z 01e679af7dbc: Pulling fs layer 2024-08-08T20:43:34.7462311Z 667d9aa101b1: Pulling fs layer 2024-08-08T20:43:34.7462649Z 529bfa1f0389: Waiting 2024-08-08T20:43:34.7463105Z 4597a1cd71c8: Pulling fs layer 2024-08-08T20:43:34.7463677Z c5585171af7c: Waiting 2024-08-08T20:43:34.7464192Z 241d45dc1ded: Waiting 2024-08-08T20:43:34.7464795Z 01e679af7dbc: Waiting 2024-08-08T20:43:34.7465421Z d5d1806bb65a: Pulling fs layer 2024-08-08T20:43:34.7466038Z 667d9aa101b1: Waiting 2024-08-08T20:43:34.7466463Z 4597a1cd71c8: Waiting 2024-08-08T20:43:34.7466839Z 2090aa744d99: Pulling fs layer 2024-08-08T20:43:34.7467323Z 508b81f2bbec: Pulling fs layer 2024-08-08T20:43:34.7467789Z d5d1806bb65a: Waiting 2024-08-08T20:43:34.7468428Z 2090aa744d99: Waiting 2024-08-08T20:43:34.7468942Z 4f4fb700ef54: Pulling fs layer 2024-08-08T20:43:34.7469385Z 6501ca0d0154: Pulling fs layer 2024-08-08T20:43:34.7469787Z 56781236accc: Pulling fs layer 2024-08-08T20:43:34.7470328Z 3794293299cd: Pulling fs layer 2024-08-08T20:43:34.7470669Z 508b81f2bbec: Waiting 2024-08-08T20:43:34.7471055Z 4f4fb700ef54: Waiting 2024-08-08T20:43:34.7471437Z 6501ca0d0154: Waiting 2024-08-08T20:43:34.7471803Z 56781236accc: Waiting 2024-08-08T20:43:34.7472262Z a4bf37ae774b: Pulling fs layer 2024-08-08T20:43:34.7472672Z 5f92aea02160: Pulling fs layer 2024-08-08T20:43:34.7473161Z 3794293299cd: Waiting 2024-08-08T20:43:34.7473474Z 5f92aea02160: Waiting 2024-08-08T20:43:34.7473914Z 3976563a126e: Pulling fs layer 2024-08-08T20:43:34.7474370Z 1457a83f8110: Pulling fs layer 2024-08-08T20:43:34.7474796Z a4bf37ae774b: Waiting 2024-08-08T20:43:34.7475363Z f4957e5566df: Pulling fs layer 2024-08-08T20:43:34.7475800Z 3976563a126e: Waiting 2024-08-08T20:43:34.7476166Z 1457a83f8110: Waiting 2024-08-08T20:43:34.7476627Z ea28e901cc2b: Pulling fs layer 2024-08-08T20:43:34.7477014Z f4957e5566df: Waiting 2024-08-08T20:43:34.7477400Z 128f963e326f: Pulling fs layer 2024-08-08T20:43:34.7477858Z 0e037b263d65: Pulling fs layer 2024-08-08T20:43:34.7478328Z 128f963e326f: Waiting 2024-08-08T20:43:34.7478757Z ea28e901cc2b: Waiting 2024-08-08T20:43:34.7479070Z 75143e4e0c77: Pulling fs layer 2024-08-08T20:43:34.7479527Z 5f2762373cea: Pulling fs layer 2024-08-08T20:43:34.7480249Z 0e037b263d65: Waiting 2024-08-08T20:43:34.7480688Z b0fdf0dc9c7d: Pulling fs layer 2024-08-08T20:43:34.7481262Z 7cf5bf391d8b: Pulling fs layer 2024-08-08T20:43:34.7481748Z c274269f5d7e: Pulling fs layer 2024-08-08T20:43:34.7482190Z 5f2762373cea: Waiting 2024-08-08T20:43:34.7482746Z 75143e4e0c77: Waiting 2024-08-08T20:43:34.7483366Z 163b636fa1af: Pulling fs layer 2024-08-08T20:43:34.7483900Z b0fdf0dc9c7d: Waiting 2024-08-08T20:43:34.7484412Z c274269f5d7e: Waiting 2024-08-08T20:43:34.7484931Z 7cf5bf391d8b: Waiting 2024-08-08T20:43:34.7485357Z 21c5c3d188ac: Pulling fs layer 2024-08-08T20:43:34.7485782Z 163b636fa1af: Waiting 2024-08-08T20:43:34.7486200Z 8befc1fd9685: Pulling fs layer 2024-08-08T20:43:34.7486652Z 6061510659f3: Pulling fs layer 2024-08-08T20:43:34.7487091Z 1baeb6c7bd59: Pulling fs layer 2024-08-08T20:43:34.7487553Z 8b31ac4f8898: Pulling fs layer 2024-08-08T20:43:34.7488083Z 64f3015ce9dd: Pulling fs layer 2024-08-08T20:43:34.7488453Z ee76d9405e95: Pulling fs layer 2024-08-08T20:43:34.7488893Z 21c5c3d188ac: Waiting 2024-08-08T20:43:34.7489276Z 6061510659f3: Waiting 2024-08-08T20:43:34.7489656Z 35b0f99cd85f: Pulling fs layer 2024-08-08T20:43:34.7490357Z 8befc1fd9685: Waiting 2024-08-08T20:43:34.7490922Z cd4355139790: Pulling fs layer 2024-08-08T20:43:34.7491351Z 1baeb6c7bd59: Waiting 2024-08-08T20:43:34.7491738Z 64f3015ce9dd: Waiting 2024-08-08T20:43:34.7492121Z ee76d9405e95: Waiting 2024-08-08T20:43:34.7492496Z 8b31ac4f8898: Waiting 2024-08-08T20:43:34.7492876Z 35b0f99cd85f: Waiting 2024-08-08T20:43:34.7493290Z 6520dd9eea8a: Pulling fs layer 2024-08-08T20:43:34.7493713Z cd4355139790: Waiting 2024-08-08T20:43:34.8961041Z cfb3d849840e: Verifying Checksum 2024-08-08T20:43:34.8961628Z cfb3d849840e: Download complete 2024-08-08T20:43:34.9862353Z ea310eb267ca: Verifying Checksum 2024-08-08T20:43:34.9862910Z ea310eb267ca: Download complete 2024-08-08T20:43:35.0575967Z 3af11d09e9cd: Verifying Checksum 2024-08-08T20:43:35.0820371Z 63e9bbe32327: Verifying Checksum 2024-08-08T20:43:35.0821345Z 63e9bbe32327: Download complete 2024-08-08T20:43:35.1647013Z 533b4aebf169: Verifying Checksum 2024-08-08T20:43:35.1647913Z 533b4aebf169: Download complete 2024-08-08T20:43:35.2334020Z 9dd75d06a091: Verifying Checksum 2024-08-08T20:43:35.2334636Z 9dd75d06a091: Download complete 2024-08-08T20:43:35.3285112Z 30bfca4dd349: Verifying Checksum 2024-08-08T20:43:35.3285718Z 30bfca4dd349: Download complete 2024-08-08T20:43:35.3751665Z 968831e596a6: Verifying Checksum 2024-08-08T20:43:35.3752742Z 968831e596a6: Download complete 2024-08-08T20:43:35.4786456Z 9ee6bdb31195: Verifying Checksum 2024-08-08T20:43:35.4787500Z 9ee6bdb31195: Download complete 2024-08-08T20:43:35.5566590Z 08be3f7f3cfb: Verifying Checksum 2024-08-08T20:43:35.5567188Z 08be3f7f3cfb: Download complete 2024-08-08T20:43:36.5210379Z 63e9bbe32327: Pull complete 2024-08-08T20:43:36.8950033Z cfb3d849840e: Pull complete 2024-08-08T20:43:38.1137526Z 968831e596a6: Pull complete 2024-08-08T20:43:38.1252446Z fd57b6413183: Verifying Checksum 2024-08-08T20:43:38.1253200Z fd57b6413183: Download complete 2024-08-08T20:43:38.1381281Z ea310eb267ca: Pull complete 2024-08-08T20:43:38.1633222Z 3af11d09e9cd: Pull complete 2024-08-08T20:43:38.2274350Z 9a1bc15482bf: Verifying Checksum 2024-08-08T20:43:38.2274968Z 9a1bc15482bf: Download complete 2024-08-08T20:43:38.3010785Z 909b1bf15ecc: Verifying Checksum 2024-08-08T20:43:38.3011438Z 909b1bf15ecc: Download complete 2024-08-08T20:43:38.4020803Z 88e7a278ede8: Verifying Checksum 2024-08-08T20:43:38.4021352Z 88e7a278ede8: Download complete 2024-08-08T20:43:39.3694621Z e5d7856b663f: Verifying Checksum 2024-08-08T20:43:39.3695233Z e5d7856b663f: Download complete 2024-08-08T20:43:39.4552125Z 00a0e1fafdad: Verifying Checksum 2024-08-08T20:43:39.4552726Z 00a0e1fafdad: Download complete 2024-08-08T20:43:39.5395419Z b137e3b1350c: Verifying Checksum 2024-08-08T20:43:39.5396098Z b137e3b1350c: Download complete 2024-08-08T20:43:39.6174359Z fc51810b1926: Verifying Checksum 2024-08-08T20:43:39.6175443Z fc51810b1926: Download complete 2024-08-08T20:43:48.7266870Z ebfec18059b9: Verifying Checksum 2024-08-08T20:43:48.7267939Z ebfec18059b9: Download complete 2024-08-08T20:43:48.8045546Z 4192f5853f54: Verifying Checksum 2024-08-08T20:43:48.8046930Z 4192f5853f54: Download complete 2024-08-08T20:43:48.8771817Z 62e5e179e864: Download complete 2024-08-08T20:43:48.9552909Z 3e326a36ef0f: Verifying Checksum 2024-08-08T20:43:48.9554035Z 3e326a36ef0f: Download complete 2024-08-08T20:43:49.0284553Z 56d2d1e08c75: Verifying Checksum 2024-08-08T20:43:49.0285294Z 56d2d1e08c75: Download complete 2024-08-08T20:43:49.1012113Z 438d85947b43: Download complete 2024-08-08T20:43:50.3860532Z 65072a4267b4: Verifying Checksum 2024-08-08T20:43:50.3861130Z 65072a4267b4: Download complete 2024-08-08T20:43:50.5014090Z c88cf310204c: Verifying Checksum 2024-08-08T20:43:50.5718847Z a54e8a071aef: Verifying Checksum 2024-08-08T20:43:50.5719475Z a54e8a071aef: Download complete 2024-08-08T20:43:50.6561379Z 37cedc534d88: Download complete 2024-08-08T20:43:50.7264995Z 33a4aacd6d0e: Verifying Checksum 2024-08-08T20:43:50.7266020Z 33a4aacd6d0e: Download complete 2024-08-08T20:43:50.8021295Z c5585171af7c: Verifying Checksum 2024-08-08T20:43:50.8021916Z c5585171af7c: Download complete 2024-08-08T20:43:55.1654898Z f956a95cc728: Verifying Checksum 2024-08-08T20:43:55.1655957Z f956a95cc728: Download complete 2024-08-08T20:43:55.2401475Z 57f752d95ca7: Verifying Checksum 2024-08-08T20:43:55.2402118Z 57f752d95ca7: Download complete 2024-08-08T20:43:55.3505319Z 529bfa1f0389: Verifying Checksum 2024-08-08T20:43:55.3506007Z 529bfa1f0389: Download complete 2024-08-08T20:43:55.7678673Z 241d45dc1ded: Verifying Checksum 2024-08-08T20:43:55.7679495Z 241d45dc1ded: Download complete 2024-08-08T20:43:55.8466464Z 01e679af7dbc: Verifying Checksum 2024-08-08T20:43:55.8467281Z 01e679af7dbc: Download complete 2024-08-08T20:43:57.1706551Z 667d9aa101b1: Verifying Checksum 2024-08-08T20:43:57.1707176Z 667d9aa101b1: Download complete 2024-08-08T20:43:57.4356457Z 4597a1cd71c8: Verifying Checksum 2024-08-08T20:43:57.4357096Z 4597a1cd71c8: Download complete 2024-08-08T20:43:57.5142323Z d5d1806bb65a: Verifying Checksum 2024-08-08T20:43:57.5143415Z d5d1806bb65a: Download complete 2024-08-08T20:43:57.5904793Z 2090aa744d99: Verifying Checksum 2024-08-08T20:43:57.5905982Z 2090aa744d99: Download complete 2024-08-08T20:43:57.6731339Z 508b81f2bbec: Download complete 2024-08-08T20:43:57.6835670Z 4f4fb700ef54: Verifying Checksum 2024-08-08T20:43:57.6836938Z 4f4fb700ef54: Download complete 2024-08-08T20:43:57.7665817Z 6501ca0d0154: Verifying Checksum 2024-08-08T20:43:57.7667025Z 6501ca0d0154: Download complete 2024-08-08T20:43:57.8533074Z 56781236accc: Download complete 2024-08-08T20:43:58.3373101Z 3794293299cd: Verifying Checksum 2024-08-08T20:43:58.3374101Z 3794293299cd: Download complete 2024-08-08T20:43:58.4191096Z a4bf37ae774b: Verifying Checksum 2024-08-08T20:43:58.4191711Z a4bf37ae774b: Download complete 2024-08-08T20:43:58.4947225Z 5f92aea02160: Verifying Checksum 2024-08-08T20:43:58.4947863Z 5f92aea02160: Download complete 2024-08-08T20:43:58.5751162Z 3976563a126e: Verifying Checksum 2024-08-08T20:43:58.5751773Z 3976563a126e: Download complete 2024-08-08T20:43:58.6566822Z 1457a83f8110: Verifying Checksum 2024-08-08T20:43:58.6567650Z 1457a83f8110: Download complete 2024-08-08T20:44:01.4596410Z 1b57ce94cad9: Verifying Checksum 2024-08-08T20:44:01.4597165Z 1b57ce94cad9: Download complete 2024-08-08T20:44:01.5299099Z ea28e901cc2b: Verifying Checksum 2024-08-08T20:44:01.6472616Z 128f963e326f: Verifying Checksum 2024-08-08T20:44:01.6477090Z 128f963e326f: Download complete 2024-08-08T20:44:01.7319593Z 0e037b263d65: Verifying Checksum 2024-08-08T20:44:01.7320747Z 0e037b263d65: Download complete 2024-08-08T20:44:01.7972576Z 75143e4e0c77: Verifying Checksum 2024-08-08T20:44:01.7973667Z 75143e4e0c77: Download complete 2024-08-08T20:44:01.8842139Z 5f2762373cea: Download complete 2024-08-08T20:44:02.0292211Z b0fdf0dc9c7d: Verifying Checksum 2024-08-08T20:44:02.0292803Z b0fdf0dc9c7d: Download complete 2024-08-08T20:44:02.0998693Z 7cf5bf391d8b: Verifying Checksum 2024-08-08T20:44:02.0999514Z 7cf5bf391d8b: Download complete 2024-08-08T20:44:02.1643413Z c274269f5d7e: Verifying Checksum 2024-08-08T20:44:02.1644524Z c274269f5d7e: Download complete 2024-08-08T20:44:02.2467671Z 163b636fa1af: Download complete 2024-08-08T20:44:02.3314023Z 21c5c3d188ac: Verifying Checksum 2024-08-08T20:44:02.3314859Z 21c5c3d188ac: Download complete 2024-08-08T20:44:02.4858399Z 8befc1fd9685: Verifying Checksum 2024-08-08T20:44:02.4859811Z 8befc1fd9685: Download complete 2024-08-08T20:44:02.5611493Z 6061510659f3: Verifying Checksum 2024-08-08T20:44:02.5612412Z 6061510659f3: Download complete 2024-08-08T20:44:03.1593409Z 1baeb6c7bd59: Verifying Checksum 2024-08-08T20:44:03.1594231Z 1baeb6c7bd59: Download complete 2024-08-08T20:44:03.2829470Z 8b31ac4f8898: Verifying Checksum 2024-08-08T20:44:03.2830527Z 8b31ac4f8898: Download complete 2024-08-08T20:44:05.3906499Z ebfec18059b9: Pull complete 2024-08-08T20:44:05.8521730Z 533b4aebf169: Pull complete 2024-08-08T20:44:06.3064416Z 9dd75d06a091: Pull complete 2024-08-08T20:44:06.7505950Z 30bfca4dd349: Pull complete 2024-08-08T20:44:07.6941944Z fbcb08d9f4bc: Verifying Checksum 2024-08-08T20:44:07.6942715Z fbcb08d9f4bc: Download complete 2024-08-08T20:44:07.7874258Z ee76d9405e95: Download complete 2024-08-08T20:44:07.9126243Z 35b0f99cd85f: Download complete 2024-08-08T20:44:08.4353355Z cd4355139790: Verifying Checksum 2024-08-08T20:44:08.4353985Z cd4355139790: Download complete 2024-08-08T20:44:08.5093059Z 6520dd9eea8a: Verifying Checksum 2024-08-08T20:44:08.5094153Z 6520dd9eea8a: Download complete 2024-08-08T20:44:17.0918246Z 64f3015ce9dd: Verifying Checksum 2024-08-08T20:44:17.0918908Z 64f3015ce9dd: Download complete 2024-08-08T20:44:17.8951764Z f4957e5566df: Verifying Checksum 2024-08-08T20:44:17.8952377Z f4957e5566df: Download complete 2024-08-08T20:44:44.7767557Z 1b57ce94cad9: Pull complete 2024-08-08T20:44:45.1759540Z 9ee6bdb31195: Pull complete 2024-08-08T20:44:45.6004299Z 08be3f7f3cfb: Pull complete 2024-08-08T20:44:56.0670410Z fd57b6413183: Pull complete 2024-08-08T20:44:56.5222800Z 9a1bc15482bf: Pull complete 2024-08-08T20:44:56.9767460Z 909b1bf15ecc: Pull complete 2024-08-08T20:44:57.4388764Z 88e7a278ede8: Pull complete 2024-08-08T20:45:00.7626882Z e5d7856b663f: Pull complete 2024-08-08T20:45:01.2084132Z 00a0e1fafdad: Pull complete 2024-08-08T20:45:01.4998640Z b137e3b1350c: Pull complete 2024-08-08T20:45:01.8738532Z fc51810b1926: Pull complete 2024-08-08T20:46:07.0561093Z fbcb08d9f4bc: Pull complete 2024-08-08T20:46:07.3432910Z 4192f5853f54: Pull complete 2024-08-08T20:46:07.6632254Z 62e5e179e864: Pull complete 2024-08-08T20:46:07.9258122Z 3e326a36ef0f: Pull complete 2024-08-08T20:46:08.2437457Z 56d2d1e08c75: Pull complete 2024-08-08T20:46:08.5949039Z 438d85947b43: Pull complete 2024-08-08T20:46:13.1214031Z 65072a4267b4: Pull complete 2024-08-08T20:46:13.4298129Z c88cf310204c: Pull complete 2024-08-08T20:46:13.7040279Z a54e8a071aef: Pull complete 2024-08-08T20:46:14.0294857Z 37cedc534d88: Pull complete 2024-08-08T20:46:14.4097405Z 33a4aacd6d0e: Pull complete 2024-08-08T20:46:14.8574286Z c5585171af7c: Pull complete 2024-08-08T20:46:25.8502489Z f956a95cc728: Pull complete 2024-08-08T20:46:26.3234893Z 57f752d95ca7: Pull complete 2024-08-08T20:46:26.7896290Z 529bfa1f0389: Pull complete 2024-08-08T20:46:28.2189268Z 241d45dc1ded: Pull complete 2024-08-08T20:46:28.6733289Z 01e679af7dbc: Pull complete 2024-08-08T20:46:29.1346418Z 667d9aa101b1: Pull complete 2024-08-08T20:46:29.8472783Z 4597a1cd71c8: Pull complete 2024-08-08T20:46:30.2744517Z d5d1806bb65a: Pull complete 2024-08-08T20:46:31.0930132Z 2090aa744d99: Pull complete 2024-08-08T20:46:31.5346789Z 508b81f2bbec: Pull complete 2024-08-08T20:46:31.9927169Z 4f4fb700ef54: Pull complete 2024-08-08T20:46:32.4581905Z 6501ca0d0154: Pull complete 2024-08-08T20:46:32.9048044Z 56781236accc: Pull complete 2024-08-08T20:46:36.1657017Z 3794293299cd: Pull complete 2024-08-08T20:46:36.6160451Z a4bf37ae774b: Pull complete 2024-08-08T20:46:37.0726678Z 5f92aea02160: Pull complete 2024-08-08T20:46:37.8018847Z 3976563a126e: Pull complete 2024-08-08T20:46:38.1070407Z 1457a83f8110: Pull complete 2024-08-08T20:47:21.3908950Z f4957e5566df: Pull complete 2024-08-08T20:47:21.8512322Z ea28e901cc2b: Pull complete 2024-08-08T20:47:22.3071981Z 128f963e326f: Pull complete 2024-08-08T20:47:23.2162118Z 0e037b263d65: Pull complete 2024-08-08T20:47:24.1190223Z 75143e4e0c77: Pull complete 2024-08-08T20:47:24.5702263Z 5f2762373cea: Pull complete 2024-08-08T20:47:25.2094187Z b0fdf0dc9c7d: Pull complete 2024-08-08T20:47:25.6626752Z 7cf5bf391d8b: Pull complete 2024-08-08T20:47:26.1182253Z c274269f5d7e: Pull complete 2024-08-08T20:47:26.5678082Z 163b636fa1af: Pull complete 2024-08-08T20:47:27.0265931Z 21c5c3d188ac: Pull complete 2024-08-08T20:47:28.8631992Z 8befc1fd9685: Pull complete 2024-08-08T20:47:29.3097079Z 6061510659f3: Pull complete 2024-08-08T20:47:32.1117852Z 1baeb6c7bd59: Pull complete 2024-08-08T20:47:32.5749654Z 8b31ac4f8898: Pull complete 2024-08-08T20:47:48.6177997Z 64f3015ce9dd: Pull complete 2024-08-08T20:47:49.0716324Z ee76d9405e95: Pull complete 2024-08-08T20:47:49.4983892Z 35b0f99cd85f: Pull complete 2024-08-08T20:47:50.3259981Z cd4355139790: Pull complete 2024-08-08T20:47:50.3472907Z 6520dd9eea8a: Pull complete 2024-08-08T20:47:50.4512432Z Digest: sha256:a898a1b237f3fdd8f48bb276749a44016150d2c25ae891805a3c665b34fc8003 2024-08-08T20:47:50.4556283Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:47:50.4592831Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:47:50.4674978Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-08T20:47:50.4676212Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-08T20:47:50.4685286Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:47:50.4686469Z env: 2024-08-08T20:47:50.4686824Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:47:50.4687244Z ##[endgroup] 2024-08-08T20:47:50.4902914Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-08T20:47:50.4903668Z with: 2024-08-08T20:47:50.4903952Z driver-version: 550.54.15 2024-08-08T20:47:50.4904399Z env: 2024-08-08T20:47:50.4904681Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:47:50.4905128Z ##[endgroup] 2024-08-08T20:47:50.4988212Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-08T20:47:50.4988938Z with: 2024-08-08T20:47:50.4989279Z timeout_minutes: 10 2024-08-08T20:47:50.4989677Z max_attempts: 3 2024-08-08T20:47:50.5035448Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-08T20:47:50.5079794Z retry_wait_seconds: 10 2024-08-08T20:47:50.5080252Z polling_interval_seconds: 1 2024-08-08T20:47:50.5080700Z warning_on_retry: true 2024-08-08T20:47:50.5081132Z continue_on_error: false 2024-08-08T20:47:50.5081544Z env: 2024-08-08T20:47:50.5081865Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:47:50.5082299Z DRIVER_VERSION: 550.54.15 2024-08-08T20:47:50.5082729Z ##[endgroup] 2024-08-08T20:47:50.6090805Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-08T20:47:50.6092310Z + pre_install_nvidia_driver_amzn2 2024-08-08T20:47:50.6093727Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-08T20:47:50.9620949Z No match for argument: nvidia-driver-latest-dkms 2024-08-08T20:47:50.9622228Z No packages marked for removal. 2024-08-08T20:47:50.9700475Z Dependencies resolved. 2024-08-08T20:47:50.9714405Z Nothing to do. 2024-08-08T20:47:50.9715249Z Complete! 2024-08-08T20:47:51.0120237Z + install_nvidia_driver_common 2024-08-08T20:47:51.0124172Z + echo 'Before installing NVIDIA driver' 2024-08-08T20:47:51.0124846Z Before installing NVIDIA driver 2024-08-08T20:47:51.0127705Z + lspci 2024-08-08T20:47:51.0357726Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-08T20:47:51.0359084Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-08T20:47:51.0360214Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-08-08T20:47:51.0361062Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-08-08T20:47:51.0361884Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-08-08T20:47:51.0362712Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-08T20:47:51.0363749Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-08T20:47:51.0364791Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-08-08T20:47:51.0365522Z + lsmod 2024-08-08T20:47:51.0402250Z Module Size Used by 2024-08-08T20:47:51.0402841Z nvidia_modeset 1351680 0 2024-08-08T20:47:51.0403498Z video 65536 1 nvidia_modeset 2024-08-08T20:47:51.0404116Z wmi 36864 1 video 2024-08-08T20:47:51.0404557Z nvidia_uvm 4706304 0 2024-08-08T20:47:51.0405228Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-08T20:47:51.0405813Z drm 602112 1 nvidia 2024-08-08T20:47:51.0406270Z drm_panel_orientation_quirks 28672 1 drm 2024-08-08T20:47:51.0407021Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-08T20:47:51.0407613Z i2c_core 106496 2 nvidia,drm 2024-08-08T20:47:51.0408092Z veth 36864 0 2024-08-08T20:47:51.0408713Z xt_conntrack 16384 1 2024-08-08T20:47:51.0409281Z nft_chain_nat 16384 3 2024-08-08T20:47:51.0409677Z xt_MASQUERADE 20480 1 2024-08-08T20:47:51.0410239Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-08T20:47:51.0410849Z nf_conntrack_netlink 57344 0 2024-08-08T20:47:51.0411584Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-08T20:47:51.0412586Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-08T20:47:51.0413408Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-08T20:47:51.0414228Z xfrm_user 57344 1 2024-08-08T20:47:51.0415048Z xfrm_algo 16384 1 xfrm_user 2024-08-08T20:47:51.0415955Z xt_addrtype 16384 2 2024-08-08T20:47:51.0416637Z nft_compat 20480 4 2024-08-08T20:47:51.0417922Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-08T20:47:51.0419321Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-08T20:47:51.0420795Z br_netfilter 36864 0 2024-08-08T20:47:51.0421737Z bridge 307200 1 br_netfilter 2024-08-08T20:47:51.0422694Z stp 16384 1 bridge 2024-08-08T20:47:51.0423263Z llc 16384 2 bridge,stp 2024-08-08T20:47:51.0424251Z overlay 167936 0 2024-08-08T20:47:51.0425298Z tls 114688 0 2024-08-08T20:47:51.0426049Z nls_ascii 16384 1 2024-08-08T20:47:51.0426514Z nls_cp437 20480 1 2024-08-08T20:47:51.0426960Z vfat 24576 1 2024-08-08T20:47:51.0427400Z sunrpc 692224 1 2024-08-08T20:47:51.0427863Z fat 86016 1 vfat 2024-08-08T20:47:51.0428342Z ata_piix 45056 0 2024-08-08T20:47:51.0428777Z ghash_clmulni_intel 16384 0 2024-08-08T20:47:51.0429262Z libata 401408 1 ata_piix 2024-08-08T20:47:51.0429758Z ena 167936 0 2024-08-08T20:47:51.0430192Z aesni_intel 393216 0 2024-08-08T20:47:51.0430683Z crypto_simd 16384 1 aesni_intel 2024-08-08T20:47:51.0431219Z ptp 36864 1 ena 2024-08-08T20:47:51.0431694Z scsi_mod 290816 1 libata 2024-08-08T20:47:51.0432314Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-08T20:47:51.0432987Z scsi_common 16384 2 scsi_mod,libata 2024-08-08T20:47:51.0433612Z pps_core 24576 1 ptp 2024-08-08T20:47:51.0434000Z i8042 45056 0 2024-08-08T20:47:51.0434465Z serio 28672 3 i8042 2024-08-08T20:47:51.0434958Z button 24576 0 2024-08-08T20:47:51.0435386Z sch_fq_codel 20480 9 2024-08-08T20:47:51.0435835Z dm_mod 188416 0 2024-08-08T20:47:51.0436280Z fuse 163840 1 2024-08-08T20:47:51.0436706Z loop 36864 0 2024-08-08T20:47:51.0437158Z configfs 57344 1 2024-08-08T20:47:51.0437603Z dax 45056 1 dm_mod 2024-08-08T20:47:51.0438091Z dmi_sysfs 20480 0 2024-08-08T20:47:51.0438544Z crc32_pclmul 16384 0 2024-08-08T20:47:51.0438980Z crc32c_intel 24576 0 2024-08-08T20:47:51.0439421Z + modinfo nvidia 2024-08-08T20:47:51.0440448Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-08T20:47:51.0441375Z alias: char-major-195-* 2024-08-08T20:47:51.0441828Z version: 550.54.15 2024-08-08T20:47:51.0442249Z supported: external 2024-08-08T20:47:51.0442662Z license: NVIDIA 2024-08-08T20:47:51.0443115Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-08T20:47:51.0443715Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-08T20:47:51.0444284Z srcversion: 833721318DA517F0C2FEC97 2024-08-08T20:47:51.0444845Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-08T20:47:51.0445465Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-08T20:47:51.0446075Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-08T20:47:51.0446675Z depends: i2c-core,drm 2024-08-08T20:47:51.0447116Z retpoline: Y 2024-08-08T20:47:51.0447487Z name: nvidia 2024-08-08T20:47:51.0448236Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-08T20:47:51.0449124Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-08T20:47:51.0449931Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-08T20:47:51.0450658Z parm: NVreg_ResmanDebugLevel:int 2024-08-08T20:47:51.0451208Z parm: NVreg_RmLogonRC:int 2024-08-08T20:47:51.0451861Z parm: NVreg_ModifyDeviceFiles:int 2024-08-08T20:47:51.0452414Z parm: NVreg_DeviceFileUID:int 2024-08-08T20:47:51.0452956Z parm: NVreg_DeviceFileGID:int 2024-08-08T20:47:51.0453500Z parm: NVreg_DeviceFileMode:int 2024-08-08T20:47:51.0454142Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-08T20:47:51.0454846Z parm: NVreg_UsePageAttributeTable:int 2024-08-08T20:47:51.0455443Z parm: NVreg_EnablePCIeGen3:int 2024-08-08T20:47:51.0455954Z parm: NVreg_EnableMSI:int 2024-08-08T20:47:51.0456460Z parm: NVreg_TCEBypassMode:int 2024-08-08T20:47:51.0457026Z parm: NVreg_EnableStreamMemOPs:int 2024-08-08T20:47:51.0457673Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-08T20:47:51.0458849Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-08T20:47:51.0459547Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-08T20:47:51.0460305Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-08T20:47:51.0461050Z parm: NVreg_DynamicPowerManagement:int 2024-08-08T20:47:51.0461815Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-08T20:47:51.0462538Z parm: NVreg_EnableGpuFirmware:int 2024-08-08T20:47:51.0463149Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-08T20:47:51.0463810Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-08T20:47:51.0464491Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-08T20:47:51.0465092Z parm: NVreg_MemoryPoolSize:int 2024-08-08T20:47:51.0465662Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-08T20:47:51.0466269Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-08T20:47:51.0466837Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-08T20:47:51.0467387Z parm: NVreg_NvLinkDisable:int 2024-08-08T20:47:51.0468010Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-08T20:47:51.0468652Z parm: NVreg_RegisterPCIDriver:int 2024-08-08T20:47:51.0469243Z parm: NVreg_EnableResizableBar:int 2024-08-08T20:47:51.0469843Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-08T20:47:51.0470445Z parm: NVreg_EnableNonblockingOpen:int 2024-08-08T20:47:51.0471048Z parm: NVreg_RegistryDwords:charp 2024-08-08T20:47:51.0471668Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-08T20:47:51.0472243Z parm: NVreg_RmMsg:charp 2024-08-08T20:47:51.0472750Z parm: NVreg_GpuBlacklist:charp 2024-08-08T20:47:51.0473327Z parm: NVreg_TemporaryFilePath:charp 2024-08-08T20:47:51.0473890Z parm: NVreg_ExcludedGpus:charp 2024-08-08T20:47:51.0474619Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-08T20:47:51.0475244Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-08T20:47:51.0475822Z parm: NVreg_ImexChannelCount:int 2024-08-08T20:47:51.0476384Z parm: rm_firmware_active:charp 2024-08-08T20:47:51.0476901Z + HAS_NVIDIA_DRIVER=0 2024-08-08T20:47:51.0477394Z ++ command -v nvidia-smi 2024-08-08T20:47:51.0477892Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-08T20:47:51.0478335Z + set +e 2024-08-08T20:47:51.0478959Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-08T20:47:51.0705072Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-08T20:47:51.0706227Z + NVIDIA_SMI_STATUS=0 2024-08-08T20:47:51.0707021Z + '[' 0 -ne 0 ']' 2024-08-08T20:47:51.0707476Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-08T20:47:51.0707941Z + HAS_NVIDIA_DRIVER=1 2024-08-08T20:47:51.0709548Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-08T20:47:51.0710581Z + set -e 2024-08-08T20:47:51.0710968Z + '[' 1 -eq 0 ']' 2024-08-08T20:47:51.0711678Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-08T20:47:51.0713349Z + post_install_nvidia_driver_common 2024-08-08T20:47:51.0713858Z + sudo modprobe nvidia 2024-08-08T20:47:51.2760887Z + echo 'After installing NVIDIA driver' 2024-08-08T20:47:51.2762012Z After installing NVIDIA driver 2024-08-08T20:47:51.2762531Z + lspci 2024-08-08T20:47:51.2990872Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-08T20:47:51.2992202Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-08T20:47:51.2993238Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-08-08T20:47:51.2994178Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-08-08T20:47:51.2994964Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-08-08T20:47:51.2995837Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-08T20:47:51.2996869Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-08T20:47:51.2997897Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-08-08T20:47:51.2998624Z + lsmod 2024-08-08T20:47:51.3019736Z Module Size Used by 2024-08-08T20:47:51.3020812Z nvidia_modeset 1351680 0 2024-08-08T20:47:51.3021818Z video 65536 1 nvidia_modeset 2024-08-08T20:47:51.3023036Z wmi 36864 1 video 2024-08-08T20:47:51.3023733Z nvidia_uvm 4706304 0 2024-08-08T20:47:51.3024293Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-08T20:47:51.3024890Z drm 602112 1 nvidia 2024-08-08T20:47:51.3025438Z drm_panel_orientation_quirks 28672 1 drm 2024-08-08T20:47:51.3026077Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-08T20:47:51.3026705Z i2c_core 106496 2 nvidia,drm 2024-08-08T20:47:51.3027231Z veth 36864 0 2024-08-08T20:47:51.3027674Z xt_conntrack 16384 1 2024-08-08T20:47:51.3028134Z nft_chain_nat 16384 3 2024-08-08T20:47:51.3028600Z xt_MASQUERADE 20480 1 2024-08-08T20:47:51.3029139Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-08T20:47:51.3029788Z nf_conntrack_netlink 57344 0 2024-08-08T20:47:51.3030512Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-08T20:47:51.3031294Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-08T20:47:51.3031869Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-08T20:47:51.3032404Z xfrm_user 57344 1 2024-08-08T20:47:51.3032871Z xfrm_algo 16384 1 xfrm_user 2024-08-08T20:47:51.3033385Z xt_addrtype 16384 2 2024-08-08T20:47:51.3033846Z nft_compat 20480 4 2024-08-08T20:47:51.3034390Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-08T20:47:51.3035381Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-08T20:47:51.3036104Z br_netfilter 36864 0 2024-08-08T20:47:51.3036636Z bridge 307200 1 br_netfilter 2024-08-08T20:47:51.3037625Z stp 16384 1 bridge 2024-08-08T20:47:51.3038760Z llc 16384 2 bridge,stp 2024-08-08T20:47:51.3039267Z overlay 167936 0 2024-08-08T20:47:51.3039703Z tls 114688 0 2024-08-08T20:47:51.3040150Z nls_ascii 16384 1 2024-08-08T20:47:51.3040598Z nls_cp437 20480 1 2024-08-08T20:47:51.3041024Z vfat 24576 1 2024-08-08T20:47:51.3041473Z sunrpc 692224 1 2024-08-08T20:47:51.3041964Z fat 86016 1 vfat 2024-08-08T20:47:51.3042488Z ata_piix 45056 0 2024-08-08T20:47:51.3042940Z ghash_clmulni_intel 16384 0 2024-08-08T20:47:51.3043407Z libata 401408 1 ata_piix 2024-08-08T20:47:51.3043810Z ena 167936 0 2024-08-08T20:47:51.3044262Z aesni_intel 393216 0 2024-08-08T20:47:51.3044739Z crypto_simd 16384 1 aesni_intel 2024-08-08T20:47:51.3045265Z ptp 36864 1 ena 2024-08-08T20:47:51.3045755Z scsi_mod 290816 1 libata 2024-08-08T20:47:51.3046365Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-08T20:47:51.3047241Z scsi_common 16384 2 scsi_mod,libata 2024-08-08T20:47:51.3047795Z pps_core 24576 1 ptp 2024-08-08T20:47:51.3048267Z i8042 45056 0 2024-08-08T20:47:51.3048705Z serio 28672 3 i8042 2024-08-08T20:47:51.3049181Z button 24576 0 2024-08-08T20:47:51.3049629Z sch_fq_codel 20480 9 2024-08-08T20:47:51.3050060Z dm_mod 188416 0 2024-08-08T20:47:51.3050502Z fuse 163840 1 2024-08-08T20:47:51.3050943Z loop 36864 0 2024-08-08T20:47:51.3051371Z configfs 57344 1 2024-08-08T20:47:51.3051839Z dax 45056 1 dm_mod 2024-08-08T20:47:51.3052329Z dmi_sysfs 20480 0 2024-08-08T20:47:51.3052790Z crc32_pclmul 16384 0 2024-08-08T20:47:51.3053220Z crc32c_intel 24576 0 2024-08-08T20:47:51.3053661Z + modinfo nvidia 2024-08-08T20:47:51.3054541Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-08T20:47:51.3055437Z alias: char-major-195-* 2024-08-08T20:47:51.3055903Z version: 550.54.15 2024-08-08T20:47:51.3056305Z supported: external 2024-08-08T20:47:51.3056714Z license: NVIDIA 2024-08-08T20:47:51.3057169Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-08T20:47:51.3057748Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-08T20:47:51.3058310Z srcversion: 833721318DA517F0C2FEC97 2024-08-08T20:47:51.3059338Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-08T20:47:51.3059945Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-08T20:47:51.3060565Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-08T20:47:51.3061199Z depends: i2c-core,drm 2024-08-08T20:47:51.3061625Z retpoline: Y 2024-08-08T20:47:51.3061997Z name: nvidia 2024-08-08T20:47:51.3062768Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-08T20:47:51.3063644Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-08T20:47:51.3064447Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-08T20:47:51.3065189Z parm: NVreg_ResmanDebugLevel:int 2024-08-08T20:47:51.3065737Z parm: NVreg_RmLogonRC:int 2024-08-08T20:47:51.3066257Z parm: NVreg_ModifyDeviceFiles:int 2024-08-08T20:47:51.3066820Z parm: NVreg_DeviceFileUID:int 2024-08-08T20:47:51.3067357Z parm: NVreg_DeviceFileGID:int 2024-08-08T20:47:51.3067882Z parm: NVreg_DeviceFileMode:int 2024-08-08T20:47:51.3068537Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-08T20:47:51.3069390Z parm: NVreg_UsePageAttributeTable:int 2024-08-08T20:47:51.3069997Z parm: NVreg_EnablePCIeGen3:int 2024-08-08T20:47:51.3070529Z parm: NVreg_EnableMSI:int 2024-08-08T20:47:51.3071041Z parm: NVreg_TCEBypassMode:int 2024-08-08T20:47:51.3071590Z parm: NVreg_EnableStreamMemOPs:int 2024-08-08T20:47:51.3072259Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-08T20:47:51.3072984Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-08T20:47:51.3073664Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-08T20:47:51.3074427Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-08T20:47:51.3075175Z parm: NVreg_DynamicPowerManagement:int 2024-08-08T20:47:51.3075924Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-08T20:47:51.3076669Z parm: NVreg_EnableGpuFirmware:int 2024-08-08T20:47:51.3077282Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-08T20:47:51.3077936Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-08T20:47:51.3078614Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-08T20:47:51.3079232Z parm: NVreg_MemoryPoolSize:int 2024-08-08T20:47:51.3079790Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-08T20:47:51.3080521Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-08T20:47:51.3081107Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-08T20:47:51.3081647Z parm: NVreg_NvLinkDisable:int 2024-08-08T20:47:51.3082274Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-08T20:47:51.3082932Z parm: NVreg_RegisterPCIDriver:int 2024-08-08T20:47:51.3083508Z parm: NVreg_EnableResizableBar:int 2024-08-08T20:47:51.3084115Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-08T20:47:51.3084735Z parm: NVreg_EnableNonblockingOpen:int 2024-08-08T20:47:51.3085320Z parm: NVreg_RegistryDwords:charp 2024-08-08T20:47:51.3085941Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-08T20:47:51.3086548Z parm: NVreg_RmMsg:charp 2024-08-08T20:47:51.3087042Z parm: NVreg_GpuBlacklist:charp 2024-08-08T20:47:51.3087626Z parm: NVreg_TemporaryFilePath:charp 2024-08-08T20:47:51.3088211Z parm: NVreg_ExcludedGpus:charp 2024-08-08T20:47:51.3088791Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-08T20:47:51.3089367Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-08T20:47:51.3089957Z parm: NVreg_ImexChannelCount:int 2024-08-08T20:47:51.3090518Z parm: rm_firmware_active:charp 2024-08-08T20:47:51.3090998Z + set +e 2024-08-08T20:47:51.3091379Z + nvidia-smi 2024-08-08T20:47:51.3246208Z Thu Aug 8 20:47:51 2024 2024-08-08T20:47:51.3247173Z +-----------------------------------------------------------------------------------------+ 2024-08-08T20:47:51.3248225Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-08T20:47:51.3249253Z |-----------------------------------------+------------------------+----------------------+ 2024-08-08T20:47:51.3250264Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-08T20:47:51.3251409Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-08T20:47:51.3252290Z | | | MIG M. | 2024-08-08T20:47:51.3252971Z |=========================================+========================+======================| 2024-08-08T20:47:51.3332987Z | 0 Tesla M60 On | 00000000:00:1E.0 Off | 0 | 2024-08-08T20:47:51.3334131Z | N/A 35C P8 15W / 150W | 0MiB / 7680MiB | 0% Default | 2024-08-08T20:47:51.3334938Z | | | N/A | 2024-08-08T20:47:51.3335809Z +-----------------------------------------+------------------------+----------------------+ 2024-08-08T20:47:51.3336645Z 2024-08-08T20:47:51.3337529Z +-----------------------------------------------------------------------------------------+ 2024-08-08T20:47:51.3338330Z | Processes: | 2024-08-08T20:47:51.3339213Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-08T20:47:51.3340081Z | ID ID Usage | 2024-08-08T20:47:51.3340788Z |=========================================================================================| 2024-08-08T20:47:51.3341592Z | No running processes found | 2024-08-08T20:47:51.3342574Z +-----------------------------------------------------------------------------------------+ 2024-08-08T20:47:51.3918959Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-08T20:47:51.4113262Z Tesla M60 2024-08-08T20:47:51.4185000Z + NVIDIA_SMI_STATUS=0 2024-08-08T20:47:51.4185541Z + '[' 0 -eq 0 ']' 2024-08-08T20:47:51.4186049Z + echo 'INFO: Ignoring allowed status 0' 2024-08-08T20:47:51.4186517Z + set -e 2024-08-08T20:47:51.4187112Z INFO: Ignoring allowed status 0 2024-08-08T20:47:51.4194015Z == Installing nvidia container toolkit for amzn2023 == 2024-08-08T20:47:51.4198105Z + sudo yum install -y yum-utils 2024-08-08T20:47:51.9538703Z Last metadata expiration check: 0:19:50 ago on Thu Aug 8 20:28:01 2024. 2024-08-08T20:47:51.9839681Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-08T20:47:52.0263961Z Dependencies resolved. 2024-08-08T20:47:52.0449203Z Nothing to do. 2024-08-08T20:47:52.0450041Z Complete! 2024-08-08T20:47:52.0901496Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-08T20:47:52.0902881Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-08T20:47:52.0904613Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-08T20:47:52.4224363Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-08T20:47:52.5042558Z + sudo yum install -y nvidia-docker2 2024-08-08T20:47:53.1041339Z nvidia-container-toolkit 8.4 kB/s | 833 B 00:00 2024-08-08T20:47:53.1341676Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-08T20:47:53.1754551Z Dependencies resolved. 2024-08-08T20:47:53.1942110Z Nothing to do. 2024-08-08T20:47:53.1942898Z Complete! 2024-08-08T20:47:53.2400481Z + sudo systemctl restart docker 2024-08-08T20:48:17.4244191Z nvidia-persistenced failed to initialize. Check syslog for more details. 2024-08-08T20:48:17.4473111Z Thu Aug 8 20:48:17 2024 2024-08-08T20:48:17.4474060Z +-----------------------------------------------------------------------------------------+ 2024-08-08T20:48:17.4475148Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-08T20:48:17.4476190Z |-----------------------------------------+------------------------+----------------------+ 2024-08-08T20:48:17.4477423Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-08T20:48:17.4478772Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-08T20:48:17.4479628Z | | | MIG M. | 2024-08-08T20:48:17.4480299Z |=========================================+========================+======================| 2024-08-08T20:48:17.4557616Z | 0 Tesla M60 On | 00000000:00:1E.0 Off | 0 | 2024-08-08T20:48:17.4558976Z | N/A 35C P8 15W / 150W | 0MiB / 7680MiB | 0% Default | 2024-08-08T20:48:17.4560375Z | | | N/A | 2024-08-08T20:48:17.4561355Z +-----------------------------------------+------------------------+----------------------+ 2024-08-08T20:48:17.4562098Z 2024-08-08T20:48:17.4562949Z +-----------------------------------------------------------------------------------------+ 2024-08-08T20:48:17.4563761Z | Processes: | 2024-08-08T20:48:17.4564654Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-08T20:48:17.4565529Z | ID ID Usage | 2024-08-08T20:48:17.4566219Z |=========================================================================================| 2024-08-08T20:48:17.4567031Z | No running processes found | 2024-08-08T20:48:17.4568039Z +-----------------------------------------------------------------------------------------+ 2024-08-08T20:48:17.6126049Z Command completed after 1 attempt(s). 2024-08-08T20:48:17.6210666Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-08T20:48:17.6211897Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-08T20:48:17.6212765Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-08T20:48:17.6213587Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-08T20:48:17.6224829Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:48:17.6225460Z env: 2024-08-08T20:48:17.6225795Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:17.6226341Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:17.6226942Z ##[endgroup] 2024-08-08T20:48:17.9689413Z Defaulting to user installation because normal site-packages is not writeable 2024-08-08T20:48:18.4849274Z Collecting psutil==5.9.1 2024-08-08T20:48:18.5338105Z Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB) 2024-08-08T20:48:18.6061442Z Collecting nvidia-ml-py==11.525.84 2024-08-08T20:48:18.6103600Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2024-08-08T20:48:18.7199385Z Installing collected packages: psutil, nvidia-ml-py 2024-08-08T20:48:18.9364341Z Successfully installed nvidia-ml-py-11.525.84 psutil-5.9.1 2024-08-08T20:48:19.0086608Z Prepare all required actions 2024-08-08T20:48:19.0087311Z Getting action download info 2024-08-08T20:48:19.1170168Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-08T20:48:19.3229978Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-08T20:48:19.6041211Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-08T20:48:19.6042398Z with: 2024-08-08T20:48:19.6043515Z name: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build 2024-08-08T20:48:19.6045054Z s3-bucket: gha-artifacts 2024-08-08T20:48:19.6045891Z env: 2024-08-08T20:48:19.6046563Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:19.6047691Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:19.6048908Z ##[endgroup] 2024-08-08T20:48:19.6131213Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-08T20:48:19.6132346Z with: 2024-08-08T20:48:19.6133542Z name: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build 2024-08-08T20:48:19.6135033Z s3-bucket: gha-artifacts 2024-08-08T20:48:19.6135907Z region: us-east-1 2024-08-08T20:48:19.6136641Z env: 2024-08-08T20:48:19.6137274Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:19.6138399Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:19.6139624Z ##[endgroup] 2024-08-08T20:48:20.2552141Z (node:79891) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-08T20:48:20.2552980Z 2024-08-08T20:48:20.2553332Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-08T20:48:20.2554299Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-08T20:48:20.2555471Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-08T20:48:20.3399943Z Found 1 objects with prefix pytorch/pytorch/10308807531/linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build/ 2024-08-08T20:48:20.3401537Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-08T20:48:35.8722824Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-08T20:48:35.8734360Z Artifact download has finished successfully 2024-08-08T20:48:35.8975329Z ##[group]Run unzip -o artifacts.zip 2024-08-08T20:48:35.8975851Z unzip -o artifacts.zip 2024-08-08T20:48:35.8983037Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:48:35.8983663Z env: 2024-08-08T20:48:35.8983982Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:35.8984545Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:35.8985160Z ##[endgroup] 2024-08-08T20:48:35.9076391Z Archive: artifacts.zip 2024-08-08T20:48:35.9105167Z creating: dist/ 2024-08-08T20:48:38.1842484Z inflating: dist/torch_no_python-2.5.0a0+gitb9d86fa-py3-none-any.whl 2024-08-08T20:48:38.7689978Z inflating: dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl 2024-08-08T20:48:38.7690905Z creating: build/custom_test_artifacts/ 2024-08-08T20:48:38.7691567Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-08T20:48:38.7692483Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-08T20:48:38.7693545Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-08T20:48:38.7700144Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-08T20:48:38.7701363Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-08T20:48:38.7702463Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-08T20:48:38.7703753Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-08T20:48:38.7705141Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-08T20:48:38.7706871Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-08T20:48:38.7708249Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-08T20:48:38.7709551Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-08T20:48:38.7710956Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-08T20:48:38.7712402Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-08T20:48:38.7713849Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-08T20:48:38.7715386Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-08T20:48:38.7716847Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-08T20:48:38.7718383Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-08T20:48:38.7720458Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-08T20:48:38.7721881Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-08T20:48:38.7723190Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-08T20:48:38.7770775Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-08T20:48:38.7819589Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-08T20:48:38.7821533Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-08T20:48:38.7876249Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-08T20:48:38.7878162Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-08T20:48:38.7880035Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-08T20:48:38.7882000Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-08T20:48:38.7883877Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-08T20:48:38.7885729Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-08T20:48:38.7887882Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-08T20:48:38.7889755Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-08T20:48:38.7891562Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-08T20:48:38.7893159Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-08T20:48:38.7894792Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-08T20:48:38.7896393Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-08T20:48:38.7897995Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-08T20:48:38.7899567Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-08T20:48:38.7901341Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-08T20:48:38.7982156Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-08T20:48:38.8072237Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-08T20:48:38.8073812Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-08T20:48:38.8075023Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-08T20:48:38.8076223Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-08T20:48:38.8077354Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-08T20:48:38.8078628Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-08T20:48:38.8079917Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-08T20:48:38.8081469Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-08T20:48:38.8082858Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-08T20:48:38.8084192Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-08T20:48:38.8085596Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-08T20:48:38.8086989Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-08T20:48:38.8088405Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-08T20:48:38.8089819Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-08T20:48:38.8091237Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-08T20:48:38.8108117Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-08T20:48:38.8279901Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-08T20:48:38.8281285Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-08T20:48:38.8282629Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-08T20:48:38.8284226Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-08T20:48:38.8285745Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-08T20:48:38.8287404Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-08T20:48:38.8288866Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-08T20:48:38.8290413Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-08T20:48:38.8291900Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-08T20:48:38.8293368Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-08T20:48:38.8294830Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-08T20:48:38.8313003Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-08T20:48:38.8412861Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-08T20:48:38.8414499Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-08T20:48:38.8416105Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-08T20:48:38.8417392Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-08T20:48:38.8418596Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-08T20:48:38.8419664Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-08T20:48:38.8420836Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-08T20:48:38.8422031Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-08T20:48:38.8423463Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-08T20:48:38.8424578Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-08T20:48:38.8568976Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-08T20:48:38.8644737Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-08T20:48:38.8645724Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-08T20:48:38.8646621Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-08T20:48:38.8647669Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-08T20:48:38.8654866Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-08T20:48:38.8656089Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-08T20:48:38.8657302Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-08T20:48:38.8658828Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-08T20:48:38.8660113Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-08T20:48:38.8661544Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-08T20:48:38.8663005Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-08T20:48:38.8664274Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-08T20:48:38.8665564Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-08T20:48:38.8667073Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-08T20:48:38.8668586Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-08T20:48:38.8670050Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-08T20:48:38.8671812Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-08T20:48:38.8673336Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-08T20:48:38.8674856Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-08T20:48:38.8676181Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-08T20:48:38.8677490Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-08T20:48:38.8724672Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-08T20:48:38.8773791Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-08T20:48:38.8775677Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-08T20:48:38.8830118Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-08T20:48:38.8832178Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-08T20:48:38.8834095Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-08T20:48:38.8836019Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-08T20:48:38.8837870Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-08T20:48:38.8839683Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-08T20:48:38.8841512Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-08T20:48:38.8843336Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-08T20:48:38.8845115Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-08T20:48:38.8846782Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-08T20:48:38.8848401Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-08T20:48:38.8849968Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-08T20:48:38.8851560Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-08T20:48:38.8853117Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-08T20:48:38.8854723Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-08T20:48:38.8935894Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-08T20:48:38.9026266Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-08T20:48:38.9027821Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-08T20:48:38.9029102Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-08T20:48:38.9030195Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-08T20:48:38.9031365Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-08T20:48:38.9032584Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-08T20:48:38.9034192Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-08T20:48:38.9035759Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-08T20:48:38.9037254Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-08T20:48:38.9038665Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-08T20:48:38.9040087Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-08T20:48:38.9041531Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-08T20:48:38.9042990Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-08T20:48:38.9044411Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-08T20:48:38.9045855Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-08T20:48:38.9062637Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-08T20:48:38.9139762Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-08T20:48:38.9141294Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-08T20:48:38.9142688Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-08T20:48:38.9143943Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-08T20:48:38.9145112Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-08T20:48:38.9146254Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-08T20:48:38.9147426Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-08T20:48:38.9148931Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-08T20:48:38.9150019Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-08T20:48:38.9151043Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-08T20:48:38.9212012Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-08T20:48:38.9213177Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-08T20:48:38.9214144Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-08T20:48:38.9215306Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-08T20:48:38.9222044Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-08T20:48:38.9223365Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-08T20:48:38.9224645Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-08T20:48:38.9226023Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-08T20:48:38.9227372Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-08T20:48:38.9228899Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-08T20:48:38.9230780Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-08T20:48:38.9232172Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-08T20:48:38.9233553Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-08T20:48:38.9235143Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-08T20:48:38.9237057Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-08T20:48:38.9238621Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-08T20:48:38.9240209Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-08T20:48:38.9242323Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-08T20:48:38.9243925Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-08T20:48:38.9245367Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-08T20:48:38.9246756Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-08T20:48:38.9293084Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-08T20:48:38.9341970Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-08T20:48:38.9344019Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-08T20:48:38.9398869Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-08T20:48:38.9400830Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-08T20:48:38.9402786Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-08T20:48:38.9404822Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-08T20:48:38.9406803Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-08T20:48:38.9408732Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-08T20:48:38.9410620Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-08T20:48:38.9412532Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-08T20:48:38.9414385Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-08T20:48:38.9416155Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-08T20:48:38.9417849Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-08T20:48:38.9419534Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-08T20:48:38.9421227Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-08T20:48:38.9422851Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-08T20:48:38.9424535Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-08T20:48:38.9504573Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-08T20:48:38.9595120Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-08T20:48:38.9597009Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-08T20:48:38.9598394Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-08T20:48:38.9599599Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-08T20:48:38.9600869Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-08T20:48:38.9602158Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-08T20:48:38.9603641Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-08T20:48:38.9605283Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-08T20:48:38.9606882Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-08T20:48:38.9608372Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-08T20:48:38.9610053Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-08T20:48:38.9611640Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-08T20:48:38.9613176Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-08T20:48:38.9614731Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-08T20:48:38.9616271Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-08T20:48:38.9617909Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-08T20:48:38.9762803Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-08T20:48:38.9764368Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-08T20:48:38.9765946Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-08T20:48:38.9767663Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-08T20:48:38.9769327Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-08T20:48:38.9770901Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-08T20:48:38.9772510Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-08T20:48:38.9774152Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-08T20:48:38.9775792Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-08T20:48:38.9777424Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-08T20:48:38.9779012Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-08T20:48:38.9796163Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-08T20:48:38.9862818Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-08T20:48:38.9864523Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-08T20:48:38.9866006Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-08T20:48:38.9867608Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-08T20:48:38.9868865Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-08T20:48:38.9870115Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-08T20:48:38.9871342Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-08T20:48:38.9872528Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-08T20:48:38.9873624Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-08T20:48:38.9874719Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-08T20:48:38.9999298Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-08T20:48:39.0050877Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-08T20:48:39.0051692Z creating: build/lib/ 2024-08-08T20:48:39.0063305Z inflating: build/lib/libpthreadpool.a 2024-08-08T20:48:39.0072992Z inflating: build/lib/libcpuinfo_internals.a 2024-08-08T20:48:39.0082790Z inflating: build/lib/libcpuinfo.a 2024-08-08T20:48:39.0083836Z inflating: build/lib/libclog.a 2024-08-08T20:48:39.0087022Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-08T20:48:39.0194024Z inflating: build/lib/libprotobuf-lite.a 2024-08-08T20:48:39.0742634Z inflating: build/lib/libprotobuf.a 2024-08-08T20:48:39.0821274Z inflating: build/lib/libgtest.a 2024-08-08T20:48:39.0830625Z inflating: build/lib/libittnotify.a 2024-08-08T20:48:39.0921197Z inflating: build/lib/libbenchmark.a 2024-08-08T20:48:39.0955018Z inflating: build/lib/libtensorpipe_uv.a 2024-08-08T20:48:39.1032850Z inflating: build/lib/libasmjit.a 2024-08-08T20:48:39.1186232Z inflating: build/lib/libgloo.a 2024-08-08T20:48:39.1301147Z inflating: build/lib/libc10.so 2024-08-08T20:48:39.1301887Z inflating: build/lib/libfoxi_loader.a 2024-08-08T20:48:39.1304093Z inflating: build/lib/libtorch_global_deps.so 2024-08-08T20:48:39.1327620Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-08T20:48:39.1935729Z inflating: build/lib/libprotoc.a 2024-08-08T20:48:39.1957502Z inflating: build/lib/libgmock.a 2024-08-08T20:48:39.1979724Z inflating: build/lib/libnnpack.a 2024-08-08T20:48:39.1980851Z inflating: build/lib/libgtest_main.a 2024-08-08T20:48:39.1981919Z inflating: build/lib/libbenchmark_main.a 2024-08-08T20:48:40.4112669Z inflating: build/lib/libdnnl.a 2024-08-08T20:48:40.4800151Z inflating: build/lib/libtensorpipe.a 2024-08-08T20:48:40.4870025Z inflating: build/lib/libc10_cuda.so 2024-08-08T20:48:40.6370313Z inflating: build/lib/libfbgemm.a 2024-08-08T20:48:40.6371053Z inflating: build/lib/libgmock_main.a 2024-08-08T20:48:40.6678697Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-08T20:48:40.7140151Z inflating: build/lib/libgloo_cuda.a 2024-08-08T20:48:40.7364151Z inflating: build/lib/libXNNPACK.a 2024-08-08T20:48:40.7415048Z inflating: build/lib/libonnx_proto.a 2024-08-08T20:48:40.8248168Z inflating: build/lib/libonnx.a 2024-08-08T20:48:43.7479445Z inflating: build/lib/libtorch_cpu.so 2024-08-08T20:48:43.7485303Z inflating: build/lib/libunbox_lib.a 2024-08-08T20:48:47.2760780Z inflating: build/lib/libtorch_cuda.so 2024-08-08T20:48:47.2762314Z inflating: build/lib/libtorch.so 2024-08-08T20:48:47.2765909Z inflating: build/lib/libc10d_cuda_test.so 2024-08-08T20:48:47.2791206Z inflating: build/lib/libjitbackend_test.so 2024-08-08T20:48:47.2878193Z inflating: build/lib/libtorchbind_test.so 2024-08-08T20:48:47.2909752Z inflating: build/lib/libaoti_custom_ops.so 2024-08-08T20:48:47.2940633Z inflating: build/lib/libbackend_with_compiler.so 2024-08-08T20:48:47.2941699Z creating: build/lib/torch/ 2024-08-08T20:48:47.2942846Z inflating: build/lib/torch/_VF.py 2024-08-08T20:48:47.2943411Z inflating: build/lib/torch/__config__.py 2024-08-08T20:48:47.2944266Z inflating: build/lib/torch/__future__.py 2024-08-08T20:48:47.2952731Z inflating: build/lib/torch/__init__.py 2024-08-08T20:48:47.2955567Z inflating: build/lib/torch/_appdirs.py 2024-08-08T20:48:47.2956414Z inflating: build/lib/torch/_classes.py 2024-08-08T20:48:47.2957362Z inflating: build/lib/torch/_compile.py 2024-08-08T20:48:47.2959275Z inflating: build/lib/torch/_custom_ops.py 2024-08-08T20:48:47.2961452Z inflating: build/lib/torch/_deploy.py 2024-08-08T20:48:47.2964962Z inflating: build/lib/torch/_guards.py 2024-08-08T20:48:47.2970856Z inflating: build/lib/torch/_jit_internal.py 2024-08-08T20:48:47.2971907Z inflating: build/lib/torch/_linalg_utils.py 2024-08-08T20:48:47.2976789Z inflating: build/lib/torch/_lobpcg.py 2024-08-08T20:48:47.2978383Z inflating: build/lib/torch/_lowrank.py 2024-08-08T20:48:47.2995009Z inflating: build/lib/torch/_meta_registrations.py 2024-08-08T20:48:47.2996055Z inflating: build/lib/torch/_namedtensor_internals.py 2024-08-08T20:48:47.3001964Z inflating: build/lib/torch/_ops.py 2024-08-08T20:48:47.3003339Z inflating: build/lib/torch/_python_dispatcher.py 2024-08-08T20:48:47.3004420Z inflating: build/lib/torch/_size_docs.py 2024-08-08T20:48:47.3005288Z inflating: build/lib/torch/_sources.py 2024-08-08T20:48:47.3006053Z inflating: build/lib/torch/_storage_docs.py 2024-08-08T20:48:47.3007026Z inflating: build/lib/torch/_streambase.py 2024-08-08T20:48:47.3013210Z inflating: build/lib/torch/_tensor.py 2024-08-08T20:48:47.3025022Z inflating: build/lib/torch/_tensor_docs.py 2024-08-08T20:48:47.3028157Z inflating: build/lib/torch/_tensor_str.py 2024-08-08T20:48:47.3062925Z inflating: build/lib/torch/_torch_docs.py 2024-08-08T20:48:47.3067022Z inflating: build/lib/torch/_utils.py 2024-08-08T20:48:47.3068750Z inflating: build/lib/torch/_utils_internal.py 2024-08-08T20:48:47.3070250Z inflating: build/lib/torch/_vmap_internals.py 2024-08-08T20:48:47.3072665Z inflating: build/lib/torch/_weights_only_unpickler.py 2024-08-08T20:48:47.3080781Z inflating: build/lib/torch/functional.py 2024-08-08T20:48:47.3084872Z inflating: build/lib/torch/hub.py 2024-08-08T20:48:47.3090104Z inflating: build/lib/torch/library.py 2024-08-08T20:48:47.3098721Z inflating: build/lib/torch/overrides.py 2024-08-08T20:48:47.3100027Z inflating: build/lib/torch/quasirandom.py 2024-08-08T20:48:47.3101820Z inflating: build/lib/torch/random.py 2024-08-08T20:48:47.3102849Z inflating: build/lib/torch/return_types.py 2024-08-08T20:48:47.3109549Z inflating: build/lib/torch/serialization.py 2024-08-08T20:48:47.3114184Z inflating: build/lib/torch/storage.py 2024-08-08T20:48:47.3115142Z inflating: build/lib/torch/torch_version.py 2024-08-08T20:48:47.3116159Z inflating: build/lib/torch/types.py 2024-08-08T20:48:47.3116993Z creating: build/lib/torch/_awaits/ 2024-08-08T20:48:47.3117991Z inflating: build/lib/torch/_awaits/__init__.py 2024-08-08T20:48:47.3119013Z creating: build/lib/torch/_custom_op/ 2024-08-08T20:48:47.3119627Z extracting: build/lib/torch/_custom_op/__init__.py 2024-08-08T20:48:47.3120572Z inflating: build/lib/torch/_custom_op/autograd.py 2024-08-08T20:48:47.3122446Z inflating: build/lib/torch/_custom_op/functional.py 2024-08-08T20:48:47.3125534Z inflating: build/lib/torch/_custom_op/impl.py 2024-08-08T20:48:47.3126264Z creating: build/lib/torch/_decomp/ 2024-08-08T20:48:47.3128071Z inflating: build/lib/torch/_decomp/__init__.py 2024-08-08T20:48:47.3142954Z inflating: build/lib/torch/_decomp/decompositions.py 2024-08-08T20:48:47.3145071Z inflating: build/lib/torch/_decomp/decompositions_for_jvp.py 2024-08-08T20:48:47.3146339Z inflating: build/lib/torch/_decomp/decompositions_for_rng.py 2024-08-08T20:48:47.3147552Z creating: build/lib/torch/_dispatch/ 2024-08-08T20:48:47.3148159Z extracting: build/lib/torch/_dispatch/__init__.py 2024-08-08T20:48:47.3149594Z inflating: build/lib/torch/_dispatch/python.py 2024-08-08T20:48:47.3150875Z creating: build/lib/torch/_dynamo/ 2024-08-08T20:48:47.3152189Z inflating: build/lib/torch/_dynamo/__init__.py 2024-08-08T20:48:47.3153487Z inflating: build/lib/torch/_dynamo/_trace_wrapped_higher_order_op.py 2024-08-08T20:48:47.3154891Z inflating: build/lib/torch/_dynamo/bytecode_analysis.py 2024-08-08T20:48:47.3159801Z inflating: build/lib/torch/_dynamo/bytecode_transformation.py 2024-08-08T20:48:47.3161479Z inflating: build/lib/torch/_dynamo/cache_size.py 2024-08-08T20:48:47.3162930Z inflating: build/lib/torch/_dynamo/callback.py 2024-08-08T20:48:47.3163950Z inflating: build/lib/torch/_dynamo/code_context.py 2024-08-08T20:48:47.3165107Z inflating: build/lib/torch/_dynamo/codegen.py 2024-08-08T20:48:47.3167451Z inflating: build/lib/torch/_dynamo/compiled_autograd.py 2024-08-08T20:48:47.3169342Z inflating: build/lib/torch/_dynamo/comptime.py 2024-08-08T20:48:47.3172578Z inflating: build/lib/torch/_dynamo/config.py 2024-08-08T20:48:47.3177214Z inflating: build/lib/torch/_dynamo/convert_frame.py 2024-08-08T20:48:47.3179228Z inflating: build/lib/torch/_dynamo/create_parameter_op.py 2024-08-08T20:48:47.3180111Z inflating: build/lib/torch/_dynamo/current_scope_id.py 2024-08-08T20:48:47.3182575Z inflating: build/lib/torch/_dynamo/debug_utils.py 2024-08-08T20:48:47.3184124Z inflating: build/lib/torch/_dynamo/decorators.py 2024-08-08T20:48:47.3185876Z inflating: build/lib/torch/_dynamo/device_interface.py 2024-08-08T20:48:47.3187366Z inflating: build/lib/torch/_dynamo/distributed.py 2024-08-08T20:48:47.3193306Z inflating: build/lib/torch/_dynamo/eval_frame.py 2024-08-08T20:48:47.3195222Z inflating: build/lib/torch/_dynamo/exc.py 2024-08-08T20:48:47.3197075Z inflating: build/lib/torch/_dynamo/external_utils.py 2024-08-08T20:48:47.3198311Z inflating: build/lib/torch/_dynamo/funcname_cache.py 2024-08-08T20:48:47.3207810Z inflating: build/lib/torch/_dynamo/guards.py 2024-08-08T20:48:47.3209065Z inflating: build/lib/torch/_dynamo/hooks.py 2024-08-08T20:48:47.3210532Z inflating: build/lib/torch/_dynamo/logging.py 2024-08-08T20:48:47.3212021Z inflating: build/lib/torch/_dynamo/mutation_guard.py 2024-08-08T20:48:47.3220357Z inflating: build/lib/torch/_dynamo/output_graph.py 2024-08-08T20:48:47.3221879Z inflating: build/lib/torch/_dynamo/polyfill.py 2024-08-08T20:48:47.3223335Z inflating: build/lib/torch/_dynamo/profiler.py 2024-08-08T20:48:47.3224449Z inflating: build/lib/torch/_dynamo/replay_record.py 2024-08-08T20:48:47.3226997Z inflating: build/lib/torch/_dynamo/resume_execution.py 2024-08-08T20:48:47.3230022Z inflating: build/lib/torch/_dynamo/side_effects.py 2024-08-08T20:48:47.3232482Z inflating: build/lib/torch/_dynamo/source.py 2024-08-08T20:48:47.3244376Z inflating: build/lib/torch/_dynamo/symbolic_convert.py 2024-08-08T20:48:47.3246120Z inflating: build/lib/torch/_dynamo/tensor_version_op.py 2024-08-08T20:48:47.3247187Z inflating: build/lib/torch/_dynamo/test_case.py 2024-08-08T20:48:47.3248673Z inflating: build/lib/torch/_dynamo/test_minifier_common.py 2024-08-08T20:48:47.3250234Z inflating: build/lib/torch/_dynamo/testing.py 2024-08-08T20:48:47.3261885Z inflating: build/lib/torch/_dynamo/trace_rules.py 2024-08-08T20:48:47.3263225Z inflating: build/lib/torch/_dynamo/types.py 2024-08-08T20:48:47.3272104Z inflating: build/lib/torch/_dynamo/utils.py 2024-08-08T20:48:47.3273503Z creating: build/lib/torch/_dynamo/backends/ 2024-08-08T20:48:47.3274985Z extracting: build/lib/torch/_dynamo/backends/__init__.py 2024-08-08T20:48:47.3276596Z inflating: build/lib/torch/_dynamo/backends/common.py 2024-08-08T20:48:47.3277806Z inflating: build/lib/torch/_dynamo/backends/cudagraphs.py 2024-08-08T20:48:47.3278950Z inflating: build/lib/torch/_dynamo/backends/debugging.py 2024-08-08T20:48:47.3281152Z inflating: build/lib/torch/_dynamo/backends/distributed.py 2024-08-08T20:48:47.3283289Z inflating: build/lib/torch/_dynamo/backends/inductor.py 2024-08-08T20:48:47.3284904Z inflating: build/lib/torch/_dynamo/backends/onnxrt.py 2024-08-08T20:48:47.3286503Z inflating: build/lib/torch/_dynamo/backends/registry.py 2024-08-08T20:48:47.3288145Z inflating: build/lib/torch/_dynamo/backends/tensorrt.py 2024-08-08T20:48:47.3289184Z inflating: build/lib/torch/_dynamo/backends/torchxla.py 2024-08-08T20:48:47.3289971Z inflating: build/lib/torch/_dynamo/backends/tvm.py 2024-08-08T20:48:47.3290612Z creating: build/lib/torch/_dynamo/repro/ 2024-08-08T20:48:47.3291276Z extracting: build/lib/torch/_dynamo/repro/__init__.py 2024-08-08T20:48:47.3292822Z inflating: build/lib/torch/_dynamo/repro/after_aot.py 2024-08-08T20:48:47.3295569Z inflating: build/lib/torch/_dynamo/repro/after_dynamo.py 2024-08-08T20:48:47.3297070Z creating: build/lib/torch/_dynamo/variables/ 2024-08-08T20:48:47.3298053Z inflating: build/lib/torch/_dynamo/variables/__init__.py 2024-08-08T20:48:47.3299608Z inflating: build/lib/torch/_dynamo/variables/base.py 2024-08-08T20:48:47.3310532Z inflating: build/lib/torch/_dynamo/variables/builder.py 2024-08-08T20:48:47.3317756Z inflating: build/lib/torch/_dynamo/variables/builtin.py 2024-08-08T20:48:47.3319512Z inflating: build/lib/torch/_dynamo/variables/constant.py 2024-08-08T20:48:47.3322831Z inflating: build/lib/torch/_dynamo/variables/ctx_manager.py 2024-08-08T20:48:47.3326287Z inflating: build/lib/torch/_dynamo/variables/dicts.py 2024-08-08T20:48:47.3328166Z inflating: build/lib/torch/_dynamo/variables/distributed.py 2024-08-08T20:48:47.3332582Z inflating: build/lib/torch/_dynamo/variables/functions.py 2024-08-08T20:48:47.3339919Z inflating: build/lib/torch/_dynamo/variables/higher_order_ops.py 2024-08-08T20:48:47.3341652Z inflating: build/lib/torch/_dynamo/variables/iter.py 2024-08-08T20:48:47.3343120Z inflating: build/lib/torch/_dynamo/variables/lazy.py 2024-08-08T20:48:47.3346330Z inflating: build/lib/torch/_dynamo/variables/lists.py 2024-08-08T20:48:47.3351420Z inflating: build/lib/torch/_dynamo/variables/misc.py 2024-08-08T20:48:47.3356348Z inflating: build/lib/torch/_dynamo/variables/nn_module.py 2024-08-08T20:48:47.3358333Z inflating: build/lib/torch/_dynamo/variables/optimizer.py 2024-08-08T20:48:47.3360506Z inflating: build/lib/torch/_dynamo/variables/script_object.py 2024-08-08T20:48:47.3361671Z inflating: build/lib/torch/_dynamo/variables/sdpa.py 2024-08-08T20:48:47.3366125Z inflating: build/lib/torch/_dynamo/variables/tensor.py 2024-08-08T20:48:47.3370477Z inflating: build/lib/torch/_dynamo/variables/torch.py 2024-08-08T20:48:47.3372375Z inflating: build/lib/torch/_dynamo/variables/torch_function.py 2024-08-08T20:48:47.3377184Z inflating: build/lib/torch/_dynamo/variables/user_defined.py 2024-08-08T20:48:47.3378244Z creating: build/lib/torch/_export/ 2024-08-08T20:48:47.3379570Z inflating: build/lib/torch/_export/__init__.py 2024-08-08T20:48:47.3384873Z inflating: build/lib/torch/_export/converter.py 2024-08-08T20:48:47.3386250Z inflating: build/lib/torch/_export/error.py 2024-08-08T20:48:47.3388475Z inflating: build/lib/torch/_export/non_strict_utils.py 2024-08-08T20:48:47.3390363Z inflating: build/lib/torch/_export/pass_base.py 2024-08-08T20:48:47.3392018Z inflating: build/lib/torch/_export/tools.py 2024-08-08T20:48:47.3394635Z inflating: build/lib/torch/_export/utils.py 2024-08-08T20:48:47.3396608Z inflating: build/lib/torch/_export/verifier.py 2024-08-08T20:48:47.3398064Z inflating: build/lib/torch/_export/wrappers.py 2024-08-08T20:48:47.3399429Z creating: build/lib/torch/_export/db/ 2024-08-08T20:48:47.3400560Z inflating: build/lib/torch/_export/db/__init__.py 2024-08-08T20:48:47.3402085Z inflating: build/lib/torch/_export/db/case.py 2024-08-08T20:48:47.3403641Z inflating: build/lib/torch/_export/db/gen_example.py 2024-08-08T20:48:47.3405508Z inflating: build/lib/torch/_export/db/logging.py 2024-08-08T20:48:47.3407020Z creating: build/lib/torch/_export/db/examples/ 2024-08-08T20:48:47.3408523Z inflating: build/lib/torch/_export/db/examples/__init__.py 2024-08-08T20:48:47.3410258Z inflating: build/lib/torch/_export/db/examples/assume_constant_result.py 2024-08-08T20:48:47.3412491Z inflating: build/lib/torch/_export/db/examples/autograd_function.py 2024-08-08T20:48:47.3414529Z inflating: build/lib/torch/_export/db/examples/class_method.py 2024-08-08T20:48:47.3416664Z inflating: build/lib/torch/_export/db/examples/cond_branch_class_method.py 2024-08-08T20:48:47.3419020Z inflating: build/lib/torch/_export/db/examples/cond_branch_nested_function.py 2024-08-08T20:48:47.3421515Z inflating: build/lib/torch/_export/db/examples/cond_branch_nonlocal_variables.py 2024-08-08T20:48:47.3423959Z inflating: build/lib/torch/_export/db/examples/cond_closed_over_variable.py 2024-08-08T20:48:47.3426120Z inflating: build/lib/torch/_export/db/examples/cond_operands.py 2024-08-08T20:48:47.3428187Z inflating: build/lib/torch/_export/db/examples/cond_predicate.py 2024-08-08T20:48:47.3430598Z inflating: build/lib/torch/_export/db/examples/constrain_as_size_example.py 2024-08-08T20:48:47.3432931Z inflating: build/lib/torch/_export/db/examples/constrain_as_value_example.py 2024-08-08T20:48:47.3435165Z inflating: build/lib/torch/_export/db/examples/decorator.py 2024-08-08T20:48:47.3436980Z inflating: build/lib/torch/_export/db/examples/dictionary.py 2024-08-08T20:48:47.3439072Z inflating: build/lib/torch/_export/db/examples/dynamic_shape_assert.py 2024-08-08T20:48:47.3441370Z inflating: build/lib/torch/_export/db/examples/dynamic_shape_constructor.py 2024-08-08T20:48:47.3443035Z inflating: build/lib/torch/_export/db/examples/dynamic_shape_if_guard.py 2024-08-08T20:48:47.3445241Z inflating: build/lib/torch/_export/db/examples/dynamic_shape_map.py 2024-08-08T20:48:47.3446509Z inflating: build/lib/torch/_export/db/examples/dynamic_shape_round.py 2024-08-08T20:48:47.3448725Z inflating: build/lib/torch/_export/db/examples/dynamic_shape_slicing.py 2024-08-08T20:48:47.3449687Z inflating: build/lib/torch/_export/db/examples/dynamic_shape_view.py 2024-08-08T20:48:47.3451377Z inflating: build/lib/torch/_export/db/examples/fn_with_kwargs.py 2024-08-08T20:48:47.3453002Z inflating: build/lib/torch/_export/db/examples/list_contains.py 2024-08-08T20:48:47.3454955Z inflating: build/lib/torch/_export/db/examples/list_unpack.py 2024-08-08T20:48:47.3456969Z inflating: build/lib/torch/_export/db/examples/model_attr_mutation.py 2024-08-08T20:48:47.3459329Z inflating: build/lib/torch/_export/db/examples/nested_function.py 2024-08-08T20:48:47.3461069Z inflating: build/lib/torch/_export/db/examples/null_context_manager.py 2024-08-08T20:48:47.3462763Z inflating: build/lib/torch/_export/db/examples/optional_input.py 2024-08-08T20:48:47.3463680Z inflating: build/lib/torch/_export/db/examples/pytree_flatten.py 2024-08-08T20:48:47.3464582Z inflating: build/lib/torch/_export/db/examples/scalar_output.py 2024-08-08T20:48:47.3465512Z inflating: build/lib/torch/_export/db/examples/specialized_attribute.py 2024-08-08T20:48:47.3466461Z inflating: build/lib/torch/_export/db/examples/static_for_loop.py 2024-08-08T20:48:47.3467333Z inflating: build/lib/torch/_export/db/examples/static_if.py 2024-08-08T20:48:47.3468179Z inflating: build/lib/torch/_export/db/examples/tensor_setattr.py 2024-08-08T20:48:47.3469124Z inflating: build/lib/torch/_export/db/examples/type_reflection_method.py 2024-08-08T20:48:47.3470123Z inflating: build/lib/torch/_export/db/examples/unsupported_operator.py 2024-08-08T20:48:47.3471097Z inflating: build/lib/torch/_export/db/examples/user_input_mutation.py 2024-08-08T20:48:47.3472323Z creating: build/lib/torch/_export/pass_infra/ 2024-08-08T20:48:47.3473824Z extracting: build/lib/torch/_export/pass_infra/__init__.py 2024-08-08T20:48:47.3475979Z inflating: build/lib/torch/_export/pass_infra/node_metadata.py 2024-08-08T20:48:47.3477129Z inflating: build/lib/torch/_export/pass_infra/proxy_value.py 2024-08-08T20:48:47.3477864Z creating: build/lib/torch/_export/passes/ 2024-08-08T20:48:47.3478538Z inflating: build/lib/torch/_export/passes/__init__.py 2024-08-08T20:48:47.3479337Z inflating: build/lib/torch/_export/passes/_node_metadata_hook.py 2024-08-08T20:48:47.3480386Z inflating: build/lib/torch/_export/passes/add_runtime_assertions_for_constraints_pass.py 2024-08-08T20:48:47.3481478Z inflating: build/lib/torch/_export/passes/collect_tracepoints_pass.py 2024-08-08T20:48:47.3483176Z inflating: build/lib/torch/_export/passes/constant_folding.py 2024-08-08T20:48:47.3484169Z inflating: build/lib/torch/_export/passes/functionalize_side_effectful_ops_pass.py 2024-08-08T20:48:47.3486032Z inflating: build/lib/torch/_export/passes/lift_constants_pass.py 2024-08-08T20:48:47.3486932Z inflating: build/lib/torch/_export/passes/remove_runtime_assertions.py 2024-08-08T20:48:47.3488546Z inflating: build/lib/torch/_export/passes/replace_autocast_with_hop_pass.py 2024-08-08T20:48:47.3491084Z inflating: build/lib/torch/_export/passes/replace_set_grad_with_hop_pass.py 2024-08-08T20:48:47.3492951Z inflating: build/lib/torch/_export/passes/replace_view_ops_with_view_copy_ops_pass.py 2024-08-08T20:48:47.3494738Z inflating: build/lib/torch/_export/passes/replace_with_hop_pass_util.py 2024-08-08T20:48:47.3495884Z creating: build/lib/torch/_export/serde/ 2024-08-08T20:48:47.3497392Z extracting: build/lib/torch/_export/serde/__init__.py 2024-08-08T20:48:47.3498396Z inflating: build/lib/torch/_export/serde/aoti_schema.py 2024-08-08T20:48:47.3499140Z inflating: build/lib/torch/_export/serde/schema.py 2024-08-08T20:48:47.3499884Z inflating: build/lib/torch/_export/serde/schema_check.py 2024-08-08T20:48:47.3500647Z inflating: build/lib/torch/_export/serde/serialize.py 2024-08-08T20:48:47.3501363Z inflating: build/lib/torch/_export/serde/union.py 2024-08-08T20:48:47.3502077Z inflating: build/lib/torch/_export/serde/schema.yaml 2024-08-08T20:48:47.3502721Z creating: build/lib/torch/_functorch/ 2024-08-08T20:48:47.3503326Z inflating: build/lib/torch/_functorch/__init__.py 2024-08-08T20:48:47.3504820Z inflating: build/lib/torch/_functorch/aot_autograd.py 2024-08-08T20:48:47.3505546Z inflating: build/lib/torch/_functorch/apis.py 2024-08-08T20:48:47.3507027Z inflating: build/lib/torch/_functorch/autograd_function.py 2024-08-08T20:48:47.3507922Z inflating: build/lib/torch/_functorch/batch_norm_replacement.py 2024-08-08T20:48:47.3509587Z inflating: build/lib/torch/_functorch/benchmark_utils.py 2024-08-08T20:48:47.3510369Z inflating: build/lib/torch/_functorch/compile_utils.py 2024-08-08T20:48:47.3511846Z inflating: build/lib/torch/_functorch/compilers.py 2024-08-08T20:48:47.3512564Z inflating: build/lib/torch/_functorch/config.py 2024-08-08T20:48:47.3513267Z inflating: build/lib/torch/_functorch/deprecated.py 2024-08-08T20:48:47.3514030Z inflating: build/lib/torch/_functorch/eager_transforms.py 2024-08-08T20:48:47.3514830Z inflating: build/lib/torch/_functorch/functional_call.py 2024-08-08T20:48:47.3515596Z inflating: build/lib/torch/_functorch/fx_minifier.py 2024-08-08T20:48:47.3516341Z inflating: build/lib/torch/_functorch/make_functional.py 2024-08-08T20:48:47.3518328Z inflating: build/lib/torch/_functorch/partitioners.py 2024-08-08T20:48:47.3520059Z inflating: build/lib/torch/_functorch/pyfunctorch.py 2024-08-08T20:48:47.3521682Z inflating: build/lib/torch/_functorch/python_key.py 2024-08-08T20:48:47.3522650Z inflating: build/lib/torch/_functorch/pytree_hacks.py 2024-08-08T20:48:47.3524635Z inflating: build/lib/torch/_functorch/top_operators_github_usage.py 2024-08-08T20:48:47.3526068Z inflating: build/lib/torch/_functorch/utils.py 2024-08-08T20:48:47.3527739Z inflating: build/lib/torch/_functorch/vmap.py 2024-08-08T20:48:47.3529474Z creating: build/lib/torch/_functorch/_aot_autograd/ 2024-08-08T20:48:47.3530288Z inflating: build/lib/torch/_functorch/_aot_autograd/__init__.py 2024-08-08T20:48:47.3532152Z inflating: build/lib/torch/_functorch/_aot_autograd/autograd_cache.py 2024-08-08T20:48:47.3536843Z inflating: build/lib/torch/_functorch/_aot_autograd/collect_metadata_analysis.py 2024-08-08T20:48:47.3538792Z inflating: build/lib/torch/_functorch/_aot_autograd/dispatch_and_compile_graph.py 2024-08-08T20:48:47.3540948Z inflating: build/lib/torch/_functorch/_aot_autograd/functional_utils.py 2024-08-08T20:48:47.3543907Z inflating: build/lib/torch/_functorch/_aot_autograd/input_output_analysis.py 2024-08-08T20:48:47.3548001Z inflating: build/lib/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py 2024-08-08T20:48:47.3549837Z inflating: build/lib/torch/_functorch/_aot_autograd/logging_utils.py 2024-08-08T20:48:47.3557991Z inflating: build/lib/torch/_functorch/_aot_autograd/runtime_wrappers.py 2024-08-08T20:48:47.3562351Z inflating: build/lib/torch/_functorch/_aot_autograd/schemas.py 2024-08-08T20:48:47.3564444Z inflating: build/lib/torch/_functorch/_aot_autograd/subclass_utils.py 2024-08-08T20:48:47.3569365Z inflating: build/lib/torch/_functorch/_aot_autograd/traced_function_transforms.py 2024-08-08T20:48:47.3571201Z inflating: build/lib/torch/_functorch/_aot_autograd/utils.py 2024-08-08T20:48:47.3572402Z creating: build/lib/torch/_higher_order_ops/ 2024-08-08T20:48:47.3573850Z inflating: build/lib/torch/_higher_order_ops/__init__.py 2024-08-08T20:48:47.3575387Z inflating: build/lib/torch/_higher_order_ops/associative_scan.py 2024-08-08T20:48:47.3576444Z inflating: build/lib/torch/_higher_order_ops/auto_functionalize.py 2024-08-08T20:48:47.3578134Z inflating: build/lib/torch/_higher_order_ops/cond.py 2024-08-08T20:48:47.3579789Z inflating: build/lib/torch/_higher_order_ops/effects.py 2024-08-08T20:48:47.3582927Z inflating: build/lib/torch/_higher_order_ops/flex_attention.py 2024-08-08T20:48:47.3584620Z inflating: build/lib/torch/_higher_order_ops/map.py 2024-08-08T20:48:47.3586211Z inflating: build/lib/torch/_higher_order_ops/out_dtype.py 2024-08-08T20:48:47.3588069Z inflating: build/lib/torch/_higher_order_ops/run_const_graph.py 2024-08-08T20:48:47.3589399Z inflating: build/lib/torch/_higher_order_ops/strict_mode.py 2024-08-08T20:48:47.3590642Z inflating: build/lib/torch/_higher_order_ops/torchbind.py 2024-08-08T20:48:47.3593991Z inflating: build/lib/torch/_higher_order_ops/triton_kernel_wrap.py 2024-08-08T20:48:47.3595649Z inflating: build/lib/torch/_higher_order_ops/utils.py 2024-08-08T20:48:47.3597481Z inflating: build/lib/torch/_higher_order_ops/while_loop.py 2024-08-08T20:48:47.3599120Z inflating: build/lib/torch/_higher_order_ops/wrap.py 2024-08-08T20:48:47.3600356Z creating: build/lib/torch/_inductor/ 2024-08-08T20:48:47.3601688Z inflating: build/lib/torch/_inductor/__init__.py 2024-08-08T20:48:47.3603025Z inflating: build/lib/torch/_inductor/aoti_eager.py 2024-08-08T20:48:47.3604539Z inflating: build/lib/torch/_inductor/async_compile.py 2024-08-08T20:48:47.3607701Z inflating: build/lib/torch/_inductor/autotune_process.py 2024-08-08T20:48:47.3609273Z inflating: build/lib/torch/_inductor/bounds.py 2024-08-08T20:48:47.3621920Z inflating: build/lib/torch/_inductor/codecache.py 2024-08-08T20:48:47.3623417Z inflating: build/lib/torch/_inductor/comm_analysis.py 2024-08-08T20:48:47.3625873Z inflating: build/lib/torch/_inductor/comms.py 2024-08-08T20:48:47.3632911Z inflating: build/lib/torch/_inductor/compile_fx.py 2024-08-08T20:48:47.3637885Z inflating: build/lib/torch/_inductor/config.py 2024-08-08T20:48:47.3639565Z inflating: build/lib/torch/_inductor/constant_folding.py 2024-08-08T20:48:47.3644782Z inflating: build/lib/torch/_inductor/cpp_builder.py 2024-08-08T20:48:47.3646304Z inflating: build/lib/torch/_inductor/cpu_vec_isa.py 2024-08-08T20:48:47.3656076Z inflating: build/lib/torch/_inductor/cudagraph_trees.py 2024-08-08T20:48:47.3657820Z inflating: build/lib/torch/_inductor/cudagraph_utils.py 2024-08-08T20:48:47.3660907Z inflating: build/lib/torch/_inductor/debug.py 2024-08-08T20:48:47.3664372Z inflating: build/lib/torch/_inductor/decomposition.py 2024-08-08T20:48:47.3667091Z inflating: build/lib/torch/_inductor/dependencies.py 2024-08-08T20:48:47.3668680Z inflating: build/lib/torch/_inductor/exc.py 2024-08-08T20:48:47.3669672Z inflating: build/lib/torch/_inductor/extern_node_serializer.py 2024-08-08T20:48:47.3671276Z inflating: build/lib/torch/_inductor/freezing.py 2024-08-08T20:48:47.3672772Z inflating: build/lib/torch/_inductor/fx_utils.py 2024-08-08T20:48:47.3679995Z inflating: build/lib/torch/_inductor/graph.py 2024-08-08T20:48:47.3681254Z inflating: build/lib/torch/_inductor/hooks.py 2024-08-08T20:48:47.3682619Z inflating: build/lib/torch/_inductor/index_propagation.py 2024-08-08T20:48:47.3684264Z inflating: build/lib/torch/_inductor/inductor_prims.py 2024-08-08T20:48:47.3705284Z inflating: build/lib/torch/_inductor/ir.py 2024-08-08T20:48:47.3707241Z inflating: build/lib/torch/_inductor/jagged_lowerings.py 2024-08-08T20:48:47.3724450Z inflating: build/lib/torch/_inductor/lowering.py 2024-08-08T20:48:47.3726162Z inflating: build/lib/torch/_inductor/metrics.py 2024-08-08T20:48:47.3730797Z inflating: build/lib/torch/_inductor/mkldnn_ir.py 2024-08-08T20:48:47.3734956Z inflating: build/lib/torch/_inductor/mkldnn_lowerings.py 2024-08-08T20:48:47.3738222Z inflating: build/lib/torch/_inductor/ops_handler.py 2024-08-08T20:48:47.3739804Z inflating: build/lib/torch/_inductor/optimize_indexing.py 2024-08-08T20:48:47.3746290Z inflating: build/lib/torch/_inductor/pattern_matcher.py 2024-08-08T20:48:47.3748145Z inflating: build/lib/torch/_inductor/quantized_lowerings.py 2024-08-08T20:48:47.3749025Z inflating: build/lib/torch/_inductor/remote_cache.py 2024-08-08T20:48:47.3759861Z inflating: build/lib/torch/_inductor/scheduler.py 2024-08-08T20:48:47.3765754Z inflating: build/lib/torch/_inductor/select_algorithm.py 2024-08-08T20:48:47.3769485Z inflating: build/lib/torch/_inductor/sizevars.py 2024-08-08T20:48:47.3771157Z inflating: build/lib/torch/_inductor/subgraph_lowering.py 2024-08-08T20:48:47.3772837Z inflating: build/lib/torch/_inductor/test_case.py 2024-08-08T20:48:47.3773581Z inflating: build/lib/torch/_inductor/test_operators.py 2024-08-08T20:48:47.3778983Z inflating: build/lib/torch/_inductor/utils.py 2024-08-08T20:48:47.3780814Z inflating: build/lib/torch/_inductor/virtualized.py 2024-08-08T20:48:47.3782717Z inflating: build/lib/torch/_inductor/wrapper_benchmark.py 2024-08-08T20:48:47.3784237Z creating: build/lib/torch/_inductor/autoheuristic/ 2024-08-08T20:48:47.3785333Z extracting: build/lib/torch/_inductor/autoheuristic/__init__.py 2024-08-08T20:48:47.3787258Z inflating: build/lib/torch/_inductor/autoheuristic/autoheuristic.py 2024-08-08T20:48:47.3789482Z inflating: build/lib/torch/_inductor/autoheuristic/autoheuristic_utils.py 2024-08-08T20:48:47.3791760Z inflating: build/lib/torch/_inductor/autoheuristic/learned_heuristic_controller.py 2024-08-08T20:48:47.3794345Z inflating: build/lib/torch/_inductor/autoheuristic/learnedheuristic_interface.py 2024-08-08T20:48:47.3796576Z creating: build/lib/torch/_inductor/autoheuristic/artifacts/ 2024-08-08T20:48:47.3797635Z inflating: build/lib/torch/_inductor/autoheuristic/artifacts/_MixedMMA100.py 2024-08-08T20:48:47.3799543Z inflating: build/lib/torch/_inductor/autoheuristic/artifacts/_PadMMA100.py 2024-08-08T20:48:47.3800584Z extracting: build/lib/torch/_inductor/autoheuristic/artifacts/__init__.py 2024-08-08T20:48:47.3801393Z creating: build/lib/torch/_inductor/codegen/ 2024-08-08T20:48:47.3802104Z extracting: build/lib/torch/_inductor/codegen/__init__.py 2024-08-08T20:48:47.3803308Z inflating: build/lib/torch/_inductor/codegen/aoti_hipify_utils.py 2024-08-08T20:48:47.3804894Z inflating: build/lib/torch/_inductor/codegen/codegen_device_driver.py 2024-08-08T20:48:47.3805752Z inflating: build/lib/torch/_inductor/codegen/common.py 2024-08-08T20:48:47.3818556Z inflating: build/lib/torch/_inductor/codegen/cpp.py 2024-08-08T20:48:47.3822470Z inflating: build/lib/torch/_inductor/codegen/cpp_gemm_template.py 2024-08-08T20:48:47.3825187Z inflating: build/lib/torch/_inductor/codegen/cpp_micro_gemm.py 2024-08-08T20:48:47.3826883Z inflating: build/lib/torch/_inductor/codegen/cpp_template.py 2024-08-08T20:48:47.3829563Z inflating: build/lib/torch/_inductor/codegen/cpp_template_kernel.py 2024-08-08T20:48:47.3832379Z inflating: build/lib/torch/_inductor/codegen/cpp_utils.py 2024-08-08T20:48:47.3842015Z inflating: build/lib/torch/_inductor/codegen/cpp_wrapper_cpu.py 2024-08-08T20:48:47.3843809Z inflating: build/lib/torch/_inductor/codegen/cpp_wrapper_cuda.py 2024-08-08T20:48:47.3845513Z inflating: build/lib/torch/_inductor/codegen/cuda_combined_scheduling.py 2024-08-08T20:48:47.3851195Z inflating: build/lib/torch/_inductor/codegen/halide.py 2024-08-08T20:48:47.3854559Z inflating: build/lib/torch/_inductor/codegen/memory_planning.py 2024-08-08T20:48:47.3856285Z inflating: build/lib/torch/_inductor/codegen/multi_kernel.py 2024-08-08T20:48:47.3864177Z inflating: build/lib/torch/_inductor/codegen/simd.py 2024-08-08T20:48:47.3874829Z inflating: build/lib/torch/_inductor/codegen/triton.py 2024-08-08T20:48:47.3879010Z inflating: build/lib/torch/_inductor/codegen/triton_combo_kernel.py 2024-08-08T20:48:47.3881047Z inflating: build/lib/torch/_inductor/codegen/triton_split_scan.py 2024-08-08T20:48:47.3882187Z inflating: build/lib/torch/_inductor/codegen/triton_utils.py 2024-08-08T20:48:47.3889065Z inflating: build/lib/torch/_inductor/codegen/wrapper.py 2024-08-08T20:48:47.3890719Z creating: build/lib/torch/_inductor/codegen/cuda/ 2024-08-08T20:48:47.3892415Z extracting: build/lib/torch/_inductor/codegen/cuda/__init__.py 2024-08-08T20:48:47.3893336Z inflating: build/lib/torch/_inductor/codegen/cuda/cuda_cpp_scheduling.py 2024-08-08T20:48:47.3894677Z inflating: build/lib/torch/_inductor/codegen/cuda/cuda_env.py 2024-08-08T20:48:47.3896356Z inflating: build/lib/torch/_inductor/codegen/cuda/cuda_kernel.py 2024-08-08T20:48:47.3897900Z inflating: build/lib/torch/_inductor/codegen/cuda/cuda_template.py 2024-08-08T20:48:47.3899667Z inflating: build/lib/torch/_inductor/codegen/cuda/cutlass_epilogue_gen.py 2024-08-08T20:48:47.3901869Z inflating: build/lib/torch/_inductor/codegen/cuda/cutlass_utils.py 2024-08-08T20:48:47.3902869Z inflating: build/lib/torch/_inductor/codegen/cuda/device_op_overrides.py 2024-08-08T20:48:47.3906672Z inflating: build/lib/torch/_inductor/codegen/cuda/gemm_template.py 2024-08-08T20:48:47.3908311Z creating: build/lib/torch/_inductor/codegen/cuda/cutlass_lib_extensions/ 2024-08-08T20:48:47.3910554Z extracting: build/lib/torch/_inductor/codegen/cuda/cutlass_lib_extensions/__init__.py 2024-08-08T20:48:47.3912643Z inflating: build/lib/torch/_inductor/codegen/cuda/cutlass_lib_extensions/gemm_operation_extensions.py 2024-08-08T20:48:47.3914849Z creating: build/lib/torch/_inductor/codegen/rocm/ 2024-08-08T20:48:47.3916333Z extracting: build/lib/torch/_inductor/codegen/rocm/__init__.py 2024-08-08T20:48:47.3917851Z inflating: build/lib/torch/_inductor/codegen/rocm/ck_template.py 2024-08-08T20:48:47.3919560Z inflating: build/lib/torch/_inductor/codegen/rocm/ck_universal_gemm_template.py 2024-08-08T20:48:47.3921817Z inflating: build/lib/torch/_inductor/codegen/rocm/compile_command.py 2024-08-08T20:48:47.3923910Z inflating: build/lib/torch/_inductor/codegen/rocm/rocm_benchmark_request.py 2024-08-08T20:48:47.3924949Z inflating: build/lib/torch/_inductor/codegen/rocm/rocm_cpp_scheduling.py 2024-08-08T20:48:47.3926576Z inflating: build/lib/torch/_inductor/codegen/rocm/rocm_kernel.py 2024-08-08T20:48:47.3928454Z inflating: build/lib/torch/_inductor/codegen/rocm/rocm_template.py 2024-08-08T20:48:47.3930766Z inflating: build/lib/torch/_inductor/codegen/rocm/rocm_template_buffer.py 2024-08-08T20:48:47.3932269Z creating: build/lib/torch/_inductor/codegen/xpu/ 2024-08-08T20:48:47.3933517Z extracting: build/lib/torch/_inductor/codegen/xpu/__init__.py 2024-08-08T20:48:47.3935581Z inflating: build/lib/torch/_inductor/codegen/xpu/device_op_overrides.py 2024-08-08T20:48:47.3936689Z inflating: build/lib/torch/_inductor/codegen/cpp_prefix.h 2024-08-08T20:48:47.3937634Z creating: build/lib/torch/_inductor/codegen/aoti_runtime/ 2024-08-08T20:48:47.3939309Z inflating: build/lib/torch/_inductor/codegen/aoti_runtime/implementation.cpp 2024-08-08T20:48:47.3941025Z inflating: build/lib/torch/_inductor/codegen/aoti_runtime/interface.cpp 2024-08-08T20:48:47.3942029Z creating: build/lib/torch/_inductor/compile_worker/ 2024-08-08T20:48:47.3943896Z extracting: build/lib/torch/_inductor/compile_worker/__init__.py 2024-08-08T20:48:47.3945430Z inflating: build/lib/torch/_inductor/compile_worker/__main__.py 2024-08-08T20:48:47.3947133Z inflating: build/lib/torch/_inductor/compile_worker/subproc_pool.py 2024-08-08T20:48:47.3948949Z inflating: build/lib/torch/_inductor/compile_worker/watchdog.py 2024-08-08T20:48:47.3949771Z creating: build/lib/torch/_inductor/fx_passes/ 2024-08-08T20:48:47.3950619Z extracting: build/lib/torch/_inductor/fx_passes/__init__.py 2024-08-08T20:48:47.3952116Z inflating: build/lib/torch/_inductor/fx_passes/b2b_gemm.py 2024-08-08T20:48:47.3952978Z inflating: build/lib/torch/_inductor/fx_passes/binary_folding.py 2024-08-08T20:48:47.3953869Z inflating: build/lib/torch/_inductor/fx_passes/ddp_fusion.py 2024-08-08T20:48:47.3954769Z inflating: build/lib/torch/_inductor/fx_passes/decompose_mem_bound_mm.py 2024-08-08T20:48:47.3956562Z inflating: build/lib/torch/_inductor/fx_passes/dedupe_symint_uses.py 2024-08-08T20:48:47.3957545Z inflating: build/lib/torch/_inductor/fx_passes/efficient_conv_bn_eval.py 2024-08-08T20:48:47.3959477Z inflating: build/lib/torch/_inductor/fx_passes/freezing_patterns.py 2024-08-08T20:48:47.3960451Z inflating: build/lib/torch/_inductor/fx_passes/fuse_attention.py 2024-08-08T20:48:47.3961405Z inflating: build/lib/torch/_inductor/fx_passes/group_batch_fusion.py 2024-08-08T20:48:47.3963451Z inflating: build/lib/torch/_inductor/fx_passes/joint_graph.py 2024-08-08T20:48:47.3964576Z inflating: build/lib/torch/_inductor/fx_passes/micro_pipeline_tp.py 2024-08-08T20:48:47.3965492Z inflating: build/lib/torch/_inductor/fx_passes/misc_patterns.py 2024-08-08T20:48:47.3968628Z inflating: build/lib/torch/_inductor/fx_passes/mkldnn_fusion.py 2024-08-08T20:48:47.3970317Z inflating: build/lib/torch/_inductor/fx_passes/numeric_utils.py 2024-08-08T20:48:47.3973096Z inflating: build/lib/torch/_inductor/fx_passes/pad_mm.py 2024-08-08T20:48:47.3977564Z inflating: build/lib/torch/_inductor/fx_passes/post_grad.py 2024-08-08T20:48:47.3980567Z inflating: build/lib/torch/_inductor/fx_passes/pre_grad.py 2024-08-08T20:48:47.3988184Z inflating: build/lib/torch/_inductor/fx_passes/quantization.py 2024-08-08T20:48:47.3990940Z inflating: build/lib/torch/_inductor/fx_passes/reinplace.py 2024-08-08T20:48:47.3992460Z inflating: build/lib/torch/_inductor/fx_passes/replace_random.py 2024-08-08T20:48:47.3998602Z inflating: build/lib/torch/_inductor/fx_passes/split_cat.py 2024-08-08T20:48:47.4000139Z creating: build/lib/torch/_inductor/fx_passes/serialized_patterns/ 2024-08-08T20:48:47.4001893Z extracting: build/lib/torch/_inductor/fx_passes/serialized_patterns/__init__.py 2024-08-08T20:48:47.4004122Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_1.py 2024-08-08T20:48:47.4006499Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_10.py 2024-08-08T20:48:47.4008937Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_11.py 2024-08-08T20:48:47.4011102Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_12.py 2024-08-08T20:48:47.4012551Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_13.py 2024-08-08T20:48:47.4014376Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_14.py 2024-08-08T20:48:47.4016387Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_15.py 2024-08-08T20:48:47.4017640Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_16.py 2024-08-08T20:48:47.4019821Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_17.py 2024-08-08T20:48:47.4021711Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_18.py 2024-08-08T20:48:47.4024237Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_19.py 2024-08-08T20:48:47.4026421Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_2.py 2024-08-08T20:48:47.4028773Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_3.py 2024-08-08T20:48:47.4031106Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_4.py 2024-08-08T20:48:47.4033148Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_5.py 2024-08-08T20:48:47.4035640Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_6.py 2024-08-08T20:48:47.4038275Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_7.py 2024-08-08T20:48:47.4040402Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_8.py 2024-08-08T20:48:47.4041577Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/_sfdp_pattern_9.py 2024-08-08T20:48:47.4043522Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/addmm_pattern.py 2024-08-08T20:48:47.4044661Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/bmm_pattern.py 2024-08-08T20:48:47.4045768Z inflating: build/lib/torch/_inductor/fx_passes/serialized_patterns/mm_pattern.py 2024-08-08T20:48:47.4047025Z creating: build/lib/torch/_inductor/kernel/ 2024-08-08T20:48:47.4048101Z inflating: build/lib/torch/_inductor/kernel/__init__.py 2024-08-08T20:48:47.4049094Z inflating: build/lib/torch/_inductor/kernel/bmm.py 2024-08-08T20:48:47.4050328Z inflating: build/lib/torch/_inductor/kernel/conv.py 2024-08-08T20:48:47.4051164Z inflating: build/lib/torch/_inductor/kernel/flex_attention.py 2024-08-08T20:48:47.4052818Z inflating: build/lib/torch/_inductor/kernel/flex_decoding.py 2024-08-08T20:48:47.4054128Z inflating: build/lib/torch/_inductor/kernel/mm.py 2024-08-08T20:48:47.4055579Z inflating: build/lib/torch/_inductor/kernel/mm_common.py 2024-08-08T20:48:47.4057147Z inflating: build/lib/torch/_inductor/kernel/mm_plus_mm.py 2024-08-08T20:48:47.4059127Z inflating: build/lib/torch/_inductor/kernel/mm_scaled.py 2024-08-08T20:48:47.4060981Z inflating: build/lib/torch/_inductor/kernel/unpack_mixed_mm.py 2024-08-08T20:48:47.4062009Z creating: build/lib/torch/_inductor/package/ 2024-08-08T20:48:47.4063585Z inflating: build/lib/torch/_inductor/package/__init__.py 2024-08-08T20:48:47.4065402Z inflating: build/lib/torch/_inductor/package/build_package.py 2024-08-08T20:48:47.4066841Z inflating: build/lib/torch/_inductor/package/package.py 2024-08-08T20:48:47.4068612Z inflating: build/lib/torch/_inductor/package/pt2_archive_constants.py 2024-08-08T20:48:47.4070247Z creating: build/lib/torch/_inductor/runtime/ 2024-08-08T20:48:47.4071656Z extracting: build/lib/torch/_inductor/runtime/__init__.py 2024-08-08T20:48:47.4072605Z inflating: build/lib/torch/_inductor/runtime/compile_tasks.py 2024-08-08T20:48:47.4074247Z inflating: build/lib/torch/_inductor/runtime/coordinate_descent_tuner.py 2024-08-08T20:48:47.4075175Z inflating: build/lib/torch/_inductor/runtime/halide_helpers.py 2024-08-08T20:48:47.4076174Z inflating: build/lib/torch/_inductor/runtime/hints.py 2024-08-08T20:48:47.4076976Z inflating: build/lib/torch/_inductor/runtime/runtime_utils.py 2024-08-08T20:48:47.4077826Z inflating: build/lib/torch/_inductor/runtime/triton_helpers.py 2024-08-08T20:48:47.4080026Z inflating: build/lib/torch/_inductor/runtime/triton_heuristics.py 2024-08-08T20:48:47.4081711Z creating: build/lib/torch/_lazy/ 2024-08-08T20:48:47.4082732Z inflating: build/lib/torch/_lazy/__init__.py 2024-08-08T20:48:47.4084128Z inflating: build/lib/torch/_lazy/closure.py 2024-08-08T20:48:47.4085540Z inflating: build/lib/torch/_lazy/computation.py 2024-08-08T20:48:47.4086754Z inflating: build/lib/torch/_lazy/config.py 2024-08-08T20:48:47.4087810Z inflating: build/lib/torch/_lazy/debug.py 2024-08-08T20:48:47.4089190Z inflating: build/lib/torch/_lazy/device_context.py 2024-08-08T20:48:47.4090950Z inflating: build/lib/torch/_lazy/extract_compiled_graph.py 2024-08-08T20:48:47.4092569Z inflating: build/lib/torch/_lazy/ir_cache.py 2024-08-08T20:48:47.4093530Z inflating: build/lib/torch/_lazy/metrics.py 2024-08-08T20:48:47.4095147Z inflating: build/lib/torch/_lazy/tensor_factory_functions.py 2024-08-08T20:48:47.4095935Z inflating: build/lib/torch/_lazy/ts_backend.py 2024-08-08T20:48:47.4096543Z creating: build/lib/torch/_library/ 2024-08-08T20:48:47.4097136Z inflating: build/lib/torch/_library/__init__.py 2024-08-08T20:48:47.4098499Z inflating: build/lib/torch/_library/autograd.py 2024-08-08T20:48:47.4099670Z inflating: build/lib/torch/_library/custom_ops.py 2024-08-08T20:48:47.4101104Z inflating: build/lib/torch/_library/fake_class_registry.py 2024-08-08T20:48:47.4102570Z inflating: build/lib/torch/_library/fake_impl.py 2024-08-08T20:48:47.4104115Z inflating: build/lib/torch/_library/infer_schema.py 2024-08-08T20:48:47.4105144Z inflating: build/lib/torch/_library/simple_registry.py 2024-08-08T20:48:47.4106550Z inflating: build/lib/torch/_library/triton.py 2024-08-08T20:48:47.4107722Z inflating: build/lib/torch/_library/utils.py 2024-08-08T20:48:47.4109073Z creating: build/lib/torch/_logging/ 2024-08-08T20:48:47.4109700Z inflating: build/lib/torch/_logging/__init__.py 2024-08-08T20:48:47.4113844Z inflating: build/lib/torch/_logging/_internal.py 2024-08-08T20:48:47.4115401Z inflating: build/lib/torch/_logging/_registrations.py 2024-08-08T20:48:47.4117064Z inflating: build/lib/torch/_logging/structured.py 2024-08-08T20:48:47.4118332Z creating: build/lib/torch/_numpy/ 2024-08-08T20:48:47.4118912Z inflating: build/lib/torch/_numpy/__init__.py 2024-08-08T20:48:47.4119629Z inflating: build/lib/torch/_numpy/_binary_ufuncs_impl.py 2024-08-08T20:48:47.4121297Z inflating: build/lib/torch/_numpy/_casting_dicts.py 2024-08-08T20:48:47.4122898Z inflating: build/lib/torch/_numpy/_dtypes.py 2024-08-08T20:48:47.4124295Z inflating: build/lib/torch/_numpy/_dtypes_impl.py 2024-08-08T20:48:47.4125368Z inflating: build/lib/torch/_numpy/_funcs.py 2024-08-08T20:48:47.4130894Z inflating: build/lib/torch/_numpy/_funcs_impl.py 2024-08-08T20:48:47.4132184Z inflating: build/lib/torch/_numpy/_getlimits.py 2024-08-08T20:48:47.4134288Z inflating: build/lib/torch/_numpy/_ndarray.py 2024-08-08T20:48:47.4135887Z inflating: build/lib/torch/_numpy/_normalizations.py 2024-08-08T20:48:47.4137668Z inflating: build/lib/torch/_numpy/_reductions_impl.py 2024-08-08T20:48:47.4139654Z inflating: build/lib/torch/_numpy/_ufuncs.py 2024-08-08T20:48:47.4141111Z inflating: build/lib/torch/_numpy/_unary_ufuncs_impl.py 2024-08-08T20:48:47.4142544Z inflating: build/lib/torch/_numpy/_util.py 2024-08-08T20:48:47.4143869Z inflating: build/lib/torch/_numpy/fft.py 2024-08-08T20:48:47.4145209Z inflating: build/lib/torch/_numpy/linalg.py 2024-08-08T20:48:47.4146588Z inflating: build/lib/torch/_numpy/random.py 2024-08-08T20:48:47.4147765Z creating: build/lib/torch/_numpy/testing/ 2024-08-08T20:48:47.4148607Z inflating: build/lib/torch/_numpy/testing/__init__.py 2024-08-08T20:48:47.4154506Z inflating: build/lib/torch/_numpy/testing/utils.py 2024-08-08T20:48:47.4155572Z creating: build/lib/torch/_prims/ 2024-08-08T20:48:47.4163013Z inflating: build/lib/torch/_prims/__init__.py 2024-08-08T20:48:47.4164468Z inflating: build/lib/torch/_prims/context.py 2024-08-08T20:48:47.4165999Z inflating: build/lib/torch/_prims/debug_prims.py 2024-08-08T20:48:47.4166699Z inflating: build/lib/torch/_prims/executor.py 2024-08-08T20:48:47.4167808Z inflating: build/lib/torch/_prims/rng_prims.py 2024-08-08T20:48:47.4168879Z creating: build/lib/torch/_prims_common/ 2024-08-08T20:48:47.4174913Z inflating: build/lib/torch/_prims_common/__init__.py 2024-08-08T20:48:47.4177011Z inflating: build/lib/torch/_prims_common/wrappers.py 2024-08-08T20:48:47.4178211Z creating: build/lib/torch/_refs/ 2024-08-08T20:48:47.4195461Z inflating: build/lib/torch/_refs/__init__.py 2024-08-08T20:48:47.4196992Z inflating: build/lib/torch/_refs/_conversions.py 2024-08-08T20:48:47.4198598Z inflating: build/lib/torch/_refs/fft.py 2024-08-08T20:48:47.4199945Z creating: build/lib/torch/_refs/linalg/ 2024-08-08T20:48:47.4201571Z inflating: build/lib/torch/_refs/linalg/__init__.py 2024-08-08T20:48:47.4203007Z creating: build/lib/torch/_refs/nn/ 2024-08-08T20:48:47.4204000Z inflating: build/lib/torch/_refs/nn/__init__.py 2024-08-08T20:48:47.4204657Z creating: build/lib/torch/_refs/nn/functional/ 2024-08-08T20:48:47.4207899Z inflating: build/lib/torch/_refs/nn/functional/__init__.py 2024-08-08T20:48:47.4209169Z creating: build/lib/torch/_refs/special/ 2024-08-08T20:48:47.4210599Z inflating: build/lib/torch/_refs/special/__init__.py 2024-08-08T20:48:47.4211448Z creating: build/lib/torch/_strobelight/ 2024-08-08T20:48:47.4212769Z extracting: build/lib/torch/_strobelight/__init__.py 2024-08-08T20:48:47.4214576Z inflating: build/lib/torch/_strobelight/cli_function_profiler.py 2024-08-08T20:48:47.4216125Z inflating: build/lib/torch/_strobelight/compile_time_profiler.py 2024-08-08T20:48:47.4217460Z creating: build/lib/torch/_subclasses/ 2024-08-08T20:48:47.4218087Z inflating: build/lib/torch/_subclasses/__init__.py 2024-08-08T20:48:47.4218864Z inflating: build/lib/torch/_subclasses/_fake_tensor_utils.py 2024-08-08T20:48:47.4220557Z inflating: build/lib/torch/_subclasses/fake_impls.py 2024-08-08T20:48:47.4229925Z inflating: build/lib/torch/_subclasses/fake_tensor.py 2024-08-08T20:48:47.4231527Z inflating: build/lib/torch/_subclasses/fake_utils.py 2024-08-08T20:48:47.4235070Z inflating: build/lib/torch/_subclasses/functional_tensor.py 2024-08-08T20:48:47.4242055Z inflating: build/lib/torch/_subclasses/meta_utils.py 2024-08-08T20:48:47.4243773Z inflating: build/lib/torch/_subclasses/schema_check_mode.py 2024-08-08T20:48:47.4245294Z creating: build/lib/torch/_vendor/ 2024-08-08T20:48:47.4246628Z extracting: build/lib/torch/_vendor/__init__.py 2024-08-08T20:48:47.4247692Z creating: build/lib/torch/_vendor/packaging/ 2024-08-08T20:48:47.4248386Z inflating: build/lib/torch/_vendor/packaging/__init__.py 2024-08-08T20:48:47.4250172Z inflating: build/lib/torch/_vendor/packaging/_structures.py 2024-08-08T20:48:47.4251257Z inflating: build/lib/torch/_vendor/packaging/version.py 2024-08-08T20:48:47.4251895Z creating: build/lib/torch/amp/ 2024-08-08T20:48:47.4252621Z inflating: build/lib/torch/amp/__init__.py 2024-08-08T20:48:47.4253908Z inflating: build/lib/torch/amp/autocast_mode.py 2024-08-08T20:48:47.4256482Z inflating: build/lib/torch/amp/grad_scaler.py 2024-08-08T20:48:47.4257840Z creating: build/lib/torch/ao/ 2024-08-08T20:48:47.4259312Z inflating: build/lib/torch/ao/__init__.py 2024-08-08T20:48:47.4260560Z creating: build/lib/torch/ao/nn/ 2024-08-08T20:48:47.4261825Z inflating: build/lib/torch/ao/nn/__init__.py 2024-08-08T20:48:47.4262912Z creating: build/lib/torch/ao/nn/intrinsic/ 2024-08-08T20:48:47.4264723Z inflating: build/lib/torch/ao/nn/intrinsic/__init__.py 2024-08-08T20:48:47.4266336Z creating: build/lib/torch/ao/nn/intrinsic/modules/ 2024-08-08T20:48:47.4267228Z inflating: build/lib/torch/ao/nn/intrinsic/modules/__init__.py 2024-08-08T20:48:47.4268110Z inflating: build/lib/torch/ao/nn/intrinsic/modules/fused.py 2024-08-08T20:48:47.4269391Z creating: build/lib/torch/ao/nn/intrinsic/qat/ 2024-08-08T20:48:47.4270979Z extracting: build/lib/torch/ao/nn/intrinsic/qat/__init__.py 2024-08-08T20:48:47.4272716Z creating: build/lib/torch/ao/nn/intrinsic/qat/modules/ 2024-08-08T20:48:47.4274571Z inflating: build/lib/torch/ao/nn/intrinsic/qat/modules/__init__.py 2024-08-08T20:48:47.4276783Z inflating: build/lib/torch/ao/nn/intrinsic/qat/modules/conv_fused.py 2024-08-08T20:48:47.4279013Z inflating: build/lib/torch/ao/nn/intrinsic/qat/modules/linear_fused.py 2024-08-08T20:48:47.4280994Z inflating: build/lib/torch/ao/nn/intrinsic/qat/modules/linear_relu.py 2024-08-08T20:48:47.4282866Z creating: build/lib/torch/ao/nn/intrinsic/quantized/ 2024-08-08T20:48:47.4284710Z inflating: build/lib/torch/ao/nn/intrinsic/quantized/__init__.py 2024-08-08T20:48:47.4286929Z creating: build/lib/torch/ao/nn/intrinsic/quantized/dynamic/ 2024-08-08T20:48:47.4289121Z extracting: build/lib/torch/ao/nn/intrinsic/quantized/dynamic/__init__.py 2024-08-08T20:48:47.4290918Z creating: build/lib/torch/ao/nn/intrinsic/quantized/dynamic/modules/ 2024-08-08T20:48:47.4293298Z inflating: build/lib/torch/ao/nn/intrinsic/quantized/dynamic/modules/__init__.py 2024-08-08T20:48:47.4296123Z inflating: build/lib/torch/ao/nn/intrinsic/quantized/dynamic/modules/linear_relu.py 2024-08-08T20:48:47.4297311Z creating: build/lib/torch/ao/nn/intrinsic/quantized/modules/ 2024-08-08T20:48:47.4298713Z inflating: build/lib/torch/ao/nn/intrinsic/quantized/modules/__init__.py 2024-08-08T20:48:47.4300188Z inflating: build/lib/torch/ao/nn/intrinsic/quantized/modules/bn_relu.py 2024-08-08T20:48:47.4302572Z inflating: build/lib/torch/ao/nn/intrinsic/quantized/modules/conv_add.py 2024-08-08T20:48:47.4303783Z inflating: build/lib/torch/ao/nn/intrinsic/quantized/modules/conv_relu.py 2024-08-08T20:48:47.4305212Z inflating: build/lib/torch/ao/nn/intrinsic/quantized/modules/linear_relu.py 2024-08-08T20:48:47.4307120Z creating: build/lib/torch/ao/nn/qat/ 2024-08-08T20:48:47.4308413Z extracting: build/lib/torch/ao/nn/qat/__init__.py 2024-08-08T20:48:47.4309079Z creating: build/lib/torch/ao/nn/qat/dynamic/ 2024-08-08T20:48:47.4310451Z extracting: build/lib/torch/ao/nn/qat/dynamic/__init__.py 2024-08-08T20:48:47.4311963Z creating: build/lib/torch/ao/nn/qat/dynamic/modules/ 2024-08-08T20:48:47.4312872Z inflating: build/lib/torch/ao/nn/qat/dynamic/modules/__init__.py 2024-08-08T20:48:47.4313781Z inflating: build/lib/torch/ao/nn/qat/dynamic/modules/linear.py 2024-08-08T20:48:47.4314524Z creating: build/lib/torch/ao/nn/qat/modules/ 2024-08-08T20:48:47.4315228Z inflating: build/lib/torch/ao/nn/qat/modules/__init__.py 2024-08-08T20:48:47.4316559Z inflating: build/lib/torch/ao/nn/qat/modules/conv.py 2024-08-08T20:48:47.4318106Z inflating: build/lib/torch/ao/nn/qat/modules/embedding_ops.py 2024-08-08T20:48:47.4319400Z inflating: build/lib/torch/ao/nn/qat/modules/linear.py 2024-08-08T20:48:47.4320989Z creating: build/lib/torch/ao/nn/quantizable/ 2024-08-08T20:48:47.4321988Z extracting: build/lib/torch/ao/nn/quantizable/__init__.py 2024-08-08T20:48:47.4322751Z creating: build/lib/torch/ao/nn/quantizable/modules/ 2024-08-08T20:48:47.4323578Z inflating: build/lib/torch/ao/nn/quantizable/modules/__init__.py 2024-08-08T20:48:47.4324752Z inflating: build/lib/torch/ao/nn/quantizable/modules/activation.py 2024-08-08T20:48:47.4326515Z inflating: build/lib/torch/ao/nn/quantizable/modules/rnn.py 2024-08-08T20:48:47.4327361Z creating: build/lib/torch/ao/nn/quantized/ 2024-08-08T20:48:47.4328753Z inflating: build/lib/torch/ao/nn/quantized/__init__.py 2024-08-08T20:48:47.4330361Z inflating: build/lib/torch/ao/nn/quantized/functional.py 2024-08-08T20:48:47.4331408Z creating: build/lib/torch/ao/nn/quantized/dynamic/ 2024-08-08T20:48:47.4332780Z extracting: build/lib/torch/ao/nn/quantized/dynamic/__init__.py 2024-08-08T20:48:47.4334676Z creating: build/lib/torch/ao/nn/quantized/dynamic/modules/ 2024-08-08T20:48:47.4336415Z inflating: build/lib/torch/ao/nn/quantized/dynamic/modules/__init__.py 2024-08-08T20:48:47.4338686Z inflating: build/lib/torch/ao/nn/quantized/dynamic/modules/conv.py 2024-08-08T20:48:47.4339799Z inflating: build/lib/torch/ao/nn/quantized/dynamic/modules/linear.py 2024-08-08T20:48:47.4341937Z inflating: build/lib/torch/ao/nn/quantized/dynamic/modules/rnn.py 2024-08-08T20:48:47.4342768Z creating: build/lib/torch/ao/nn/quantized/modules/ 2024-08-08T20:48:47.4343535Z inflating: build/lib/torch/ao/nn/quantized/modules/__init__.py 2024-08-08T20:48:47.4345434Z inflating: build/lib/torch/ao/nn/quantized/modules/activation.py 2024-08-08T20:48:47.4346759Z inflating: build/lib/torch/ao/nn/quantized/modules/batchnorm.py 2024-08-08T20:48:47.4349255Z inflating: build/lib/torch/ao/nn/quantized/modules/conv.py 2024-08-08T20:48:47.4350864Z inflating: build/lib/torch/ao/nn/quantized/modules/dropout.py 2024-08-08T20:48:47.4352528Z inflating: build/lib/torch/ao/nn/quantized/modules/embedding_ops.py 2024-08-08T20:48:47.4354723Z inflating: build/lib/torch/ao/nn/quantized/modules/functional_modules.py 2024-08-08T20:48:47.4356536Z inflating: build/lib/torch/ao/nn/quantized/modules/linear.py 2024-08-08T20:48:47.4357795Z inflating: build/lib/torch/ao/nn/quantized/modules/normalization.py 2024-08-08T20:48:47.4359345Z inflating: build/lib/torch/ao/nn/quantized/modules/rnn.py 2024-08-08T20:48:47.4360604Z inflating: build/lib/torch/ao/nn/quantized/modules/utils.py 2024-08-08T20:48:47.4361390Z creating: build/lib/torch/ao/nn/quantized/reference/ 2024-08-08T20:48:47.4362225Z inflating: build/lib/torch/ao/nn/quantized/reference/__init__.py 2024-08-08T20:48:47.4363334Z creating: build/lib/torch/ao/nn/quantized/reference/modules/ 2024-08-08T20:48:47.4365252Z inflating: build/lib/torch/ao/nn/quantized/reference/modules/__init__.py 2024-08-08T20:48:47.4366302Z inflating: build/lib/torch/ao/nn/quantized/reference/modules/conv.py 2024-08-08T20:48:47.4368115Z inflating: build/lib/torch/ao/nn/quantized/reference/modules/linear.py 2024-08-08T20:48:47.4369117Z inflating: build/lib/torch/ao/nn/quantized/reference/modules/rnn.py 2024-08-08T20:48:47.4370119Z inflating: build/lib/torch/ao/nn/quantized/reference/modules/sparse.py 2024-08-08T20:48:47.4371592Z inflating: build/lib/torch/ao/nn/quantized/reference/modules/utils.py 2024-08-08T20:48:47.4372713Z creating: build/lib/torch/ao/nn/sparse/ 2024-08-08T20:48:47.4373820Z extracting: build/lib/torch/ao/nn/sparse/__init__.py 2024-08-08T20:48:47.4374888Z creating: build/lib/torch/ao/nn/sparse/quantized/ 2024-08-08T20:48:47.4375663Z inflating: build/lib/torch/ao/nn/sparse/quantized/__init__.py 2024-08-08T20:48:47.4376521Z inflating: build/lib/torch/ao/nn/sparse/quantized/linear.py 2024-08-08T20:48:47.4377360Z inflating: build/lib/torch/ao/nn/sparse/quantized/utils.py 2024-08-08T20:48:47.4378162Z creating: build/lib/torch/ao/nn/sparse/quantized/dynamic/ 2024-08-08T20:48:47.4380079Z inflating: build/lib/torch/ao/nn/sparse/quantized/dynamic/__init__.py 2024-08-08T20:48:47.4381565Z inflating: build/lib/torch/ao/nn/sparse/quantized/dynamic/linear.py 2024-08-08T20:48:47.4382886Z creating: build/lib/torch/ao/ns/ 2024-08-08T20:48:47.4384107Z extracting: build/lib/torch/ao/ns/__init__.py 2024-08-08T20:48:47.4384797Z inflating: build/lib/torch/ao/ns/_numeric_suite.py 2024-08-08T20:48:47.4385527Z inflating: build/lib/torch/ao/ns/_numeric_suite_fx.py 2024-08-08T20:48:47.4386912Z creating: build/lib/torch/ao/ns/fx/ 2024-08-08T20:48:47.4387824Z extracting: build/lib/torch/ao/ns/fx/__init__.py 2024-08-08T20:48:47.4389750Z inflating: build/lib/torch/ao/ns/fx/graph_matcher.py 2024-08-08T20:48:47.4391201Z inflating: build/lib/torch/ao/ns/fx/graph_passes.py 2024-08-08T20:48:47.4392757Z inflating: build/lib/torch/ao/ns/fx/mappings.py 2024-08-08T20:48:47.4393893Z inflating: build/lib/torch/ao/ns/fx/n_shadows_utils.py 2024-08-08T20:48:47.4394589Z inflating: build/lib/torch/ao/ns/fx/ns_types.py 2024-08-08T20:48:47.4395708Z inflating: build/lib/torch/ao/ns/fx/pattern_utils.py 2024-08-08T20:48:47.4397332Z inflating: build/lib/torch/ao/ns/fx/qconfig_multi_mapping.py 2024-08-08T20:48:47.4398498Z inflating: build/lib/torch/ao/ns/fx/utils.py 2024-08-08T20:48:47.4400009Z inflating: build/lib/torch/ao/ns/fx/weight_utils.py 2024-08-08T20:48:47.4400727Z creating: build/lib/torch/ao/pruning/ 2024-08-08T20:48:47.4401570Z inflating: build/lib/torch/ao/pruning/__init__.py 2024-08-08T20:48:47.4402898Z inflating: build/lib/torch/ao/pruning/_mappings.py 2024-08-08T20:48:47.4404457Z creating: build/lib/torch/ao/pruning/_experimental/ 2024-08-08T20:48:47.4406255Z extracting: build/lib/torch/ao/pruning/_experimental/__init__.py 2024-08-08T20:48:47.4408630Z creating: build/lib/torch/ao/pruning/_experimental/activation_sparsifier/ 2024-08-08T20:48:47.4410918Z extracting: build/lib/torch/ao/pruning/_experimental/activation_sparsifier/__init__.py 2024-08-08T20:48:47.4413241Z inflating: build/lib/torch/ao/pruning/_experimental/activation_sparsifier/activation_sparsifier.py 2024-08-08T20:48:47.4434983Z creating: build/lib/torch/ao/pruning/_experimental/data_scheduler/ 2024-08-08T20:48:47.4436098Z inflating: build/lib/torch/ao/pruning/_experimental/data_scheduler/__init__.py 2024-08-08T20:48:47.4437293Z inflating: build/lib/torch/ao/pruning/_experimental/data_scheduler/base_data_scheduler.py 2024-08-08T20:48:47.4438411Z creating: build/lib/torch/ao/pruning/_experimental/data_sparsifier/ 2024-08-08T20:48:47.4439462Z inflating: build/lib/torch/ao/pruning/_experimental/data_sparsifier/__init__.py 2024-08-08T20:48:47.4441184Z inflating: build/lib/torch/ao/pruning/_experimental/data_sparsifier/base_data_sparsifier.py 2024-08-08T20:48:47.4443560Z inflating: build/lib/torch/ao/pruning/_experimental/data_sparsifier/data_norm_sparsifier.py 2024-08-08T20:48:47.4445775Z inflating: build/lib/torch/ao/pruning/_experimental/data_sparsifier/quantization_utils.py 2024-08-08T20:48:47.4446948Z creating: build/lib/torch/ao/pruning/_experimental/data_sparsifier/lightning/ 2024-08-08T20:48:47.4448115Z extracting: build/lib/torch/ao/pruning/_experimental/data_sparsifier/lightning/__init__.py 2024-08-08T20:48:47.4449360Z creating: build/lib/torch/ao/pruning/_experimental/data_sparsifier/lightning/callbacks/ 2024-08-08T20:48:47.4450665Z extracting: build/lib/torch/ao/pruning/_experimental/data_sparsifier/lightning/callbacks/__init__.py 2024-08-08T20:48:47.4452175Z inflating: build/lib/torch/ao/pruning/_experimental/data_sparsifier/lightning/callbacks/_data_sparstity_utils.py 2024-08-08T20:48:47.4453670Z inflating: build/lib/torch/ao/pruning/_experimental/data_sparsifier/lightning/callbacks/data_sparsity.py 2024-08-08T20:48:47.4454820Z creating: build/lib/torch/ao/pruning/_experimental/pruner/ 2024-08-08T20:48:47.4455759Z inflating: build/lib/torch/ao/pruning/_experimental/pruner/FPGM_pruner.py 2024-08-08T20:48:47.4456789Z inflating: build/lib/torch/ao/pruning/_experimental/pruner/__init__.py 2024-08-08T20:48:47.4457942Z inflating: build/lib/torch/ao/pruning/_experimental/pruner/base_structured_sparsifier.py 2024-08-08T20:48:47.4459624Z inflating: build/lib/torch/ao/pruning/_experimental/pruner/lstm_saliency_pruner.py 2024-08-08T20:48:47.4460753Z inflating: build/lib/torch/ao/pruning/_experimental/pruner/match_utils.py 2024-08-08T20:48:47.4461827Z inflating: build/lib/torch/ao/pruning/_experimental/pruner/parametrization.py 2024-08-08T20:48:47.4462943Z inflating: build/lib/torch/ao/pruning/_experimental/pruner/prune_functions.py 2024-08-08T20:48:47.4464286Z inflating: build/lib/torch/ao/pruning/_experimental/pruner/saliency_pruner.py 2024-08-08T20:48:47.4465136Z creating: build/lib/torch/ao/pruning/scheduler/ 2024-08-08T20:48:47.4465920Z extracting: build/lib/torch/ao/pruning/scheduler/__init__.py 2024-08-08T20:48:47.4466828Z inflating: build/lib/torch/ao/pruning/scheduler/base_scheduler.py 2024-08-08T20:48:47.4467801Z inflating: build/lib/torch/ao/pruning/scheduler/cubic_scheduler.py 2024-08-08T20:48:47.4468767Z inflating: build/lib/torch/ao/pruning/scheduler/lambda_scheduler.py 2024-08-08T20:48:47.4469574Z creating: build/lib/torch/ao/pruning/sparsifier/ 2024-08-08T20:48:47.4470358Z extracting: build/lib/torch/ao/pruning/sparsifier/__init__.py 2024-08-08T20:48:47.4471272Z inflating: build/lib/torch/ao/pruning/sparsifier/base_sparsifier.py 2024-08-08T20:48:47.4472349Z inflating: build/lib/torch/ao/pruning/sparsifier/nearly_diagonal_sparsifier.py 2024-08-08T20:48:47.4473367Z inflating: build/lib/torch/ao/pruning/sparsifier/utils.py 2024-08-08T20:48:47.4474287Z inflating: build/lib/torch/ao/pruning/sparsifier/weight_norm_sparsifier.py 2024-08-08T20:48:47.4475286Z creating: build/lib/torch/ao/quantization/ 2024-08-08T20:48:47.4475993Z inflating: build/lib/torch/ao/quantization/__init__.py 2024-08-08T20:48:47.4476809Z inflating: build/lib/torch/ao/quantization/_correct_bias.py 2024-08-08T20:48:47.4477600Z inflating: build/lib/torch/ao/quantization/_equalize.py 2024-08-08T20:48:47.4478492Z inflating: build/lib/torch/ao/quantization/_learnable_fake_quantize.py 2024-08-08T20:48:47.4479411Z inflating: build/lib/torch/ao/quantization/fake_quantize.py 2024-08-08T20:48:47.4480251Z inflating: build/lib/torch/ao/quantization/fuse_modules.py 2024-08-08T20:48:47.4481172Z inflating: build/lib/torch/ao/quantization/fuser_method_mappings.py 2024-08-08T20:48:47.4482031Z inflating: build/lib/torch/ao/quantization/observer.py 2024-08-08T20:48:47.4482784Z inflating: build/lib/torch/ao/quantization/qconfig.py 2024-08-08T20:48:47.4483623Z inflating: build/lib/torch/ao/quantization/qconfig_mapping.py 2024-08-08T20:48:47.4484465Z inflating: build/lib/torch/ao/quantization/quant_type.py 2024-08-08T20:48:47.4485320Z inflating: build/lib/torch/ao/quantization/quantization_mappings.py 2024-08-08T20:48:47.4486242Z inflating: build/lib/torch/ao/quantization/quantize.py 2024-08-08T20:48:47.4487034Z inflating: build/lib/torch/ao/quantization/quantize_fx.py 2024-08-08T20:48:47.4487809Z inflating: build/lib/torch/ao/quantization/quantize_jit.py 2024-08-08T20:48:47.4488652Z inflating: build/lib/torch/ao/quantization/quantize_pt2e.py 2024-08-08T20:48:47.4489466Z inflating: build/lib/torch/ao/quantization/stubs.py 2024-08-08T20:48:47.4490237Z inflating: build/lib/torch/ao/quantization/utils.py 2024-08-08T20:48:47.4490970Z creating: build/lib/torch/ao/quantization/backend_config/ 2024-08-08T20:48:47.4491885Z inflating: build/lib/torch/ao/quantization/backend_config/__init__.py 2024-08-08T20:48:47.4493035Z inflating: build/lib/torch/ao/quantization/backend_config/_common_operator_config_utils.py 2024-08-08T20:48:47.4494226Z inflating: build/lib/torch/ao/quantization/backend_config/_qnnpack_pt2e.py 2024-08-08T20:48:47.4495353Z inflating: build/lib/torch/ao/quantization/backend_config/backend_config.py 2024-08-08T20:48:47.4496428Z inflating: build/lib/torch/ao/quantization/backend_config/executorch.py 2024-08-08T20:48:47.4497408Z inflating: build/lib/torch/ao/quantization/backend_config/fbgemm.py 2024-08-08T20:48:47.4498363Z inflating: build/lib/torch/ao/quantization/backend_config/native.py 2024-08-08T20:48:47.4499451Z extracting: build/lib/torch/ao/quantization/backend_config/observation_type.py 2024-08-08T20:48:47.4500495Z inflating: build/lib/torch/ao/quantization/backend_config/onednn.py 2024-08-08T20:48:47.4501463Z inflating: build/lib/torch/ao/quantization/backend_config/qnnpack.py 2024-08-08T20:48:47.4502607Z inflating: build/lib/torch/ao/quantization/backend_config/tensorrt.py 2024-08-08T20:48:47.4503612Z inflating: build/lib/torch/ao/quantization/backend_config/utils.py 2024-08-08T20:48:47.4504523Z inflating: build/lib/torch/ao/quantization/backend_config/x86.py 2024-08-08T20:48:47.4505316Z creating: build/lib/torch/ao/quantization/fx/ 2024-08-08T20:48:47.4506019Z inflating: build/lib/torch/ao/quantization/fx/__init__.py 2024-08-08T20:48:47.4506819Z inflating: build/lib/torch/ao/quantization/fx/_decomposed.py 2024-08-08T20:48:47.4507650Z inflating: build/lib/torch/ao/quantization/fx/_equalize.py 2024-08-08T20:48:47.4508853Z inflating: build/lib/torch/ao/quantization/fx/_lower_to_native_backend.py 2024-08-08T20:48:47.4514510Z inflating: build/lib/torch/ao/quantization/fx/convert.py 2024-08-08T20:48:47.4516774Z inflating: build/lib/torch/ao/quantization/fx/custom_config.py 2024-08-08T20:48:47.4517914Z inflating: build/lib/torch/ao/quantization/fx/fuse.py 2024-08-08T20:48:47.4519289Z inflating: build/lib/torch/ao/quantization/fx/fuse_handler.py 2024-08-08T20:48:47.4520869Z inflating: build/lib/torch/ao/quantization/fx/graph_module.py 2024-08-08T20:48:47.4522020Z inflating: build/lib/torch/ao/quantization/fx/lower_to_fbgemm.py 2024-08-08T20:48:47.4522970Z inflating: build/lib/torch/ao/quantization/fx/lower_to_qnnpack.py 2024-08-08T20:48:47.4524410Z inflating: build/lib/torch/ao/quantization/fx/lstm_utils.py 2024-08-08T20:48:47.4526253Z inflating: build/lib/torch/ao/quantization/fx/match_utils.py 2024-08-08T20:48:47.4527311Z inflating: build/lib/torch/ao/quantization/fx/pattern_utils.py 2024-08-08T20:48:47.4535541Z inflating: build/lib/torch/ao/quantization/fx/prepare.py 2024-08-08T20:48:47.4537811Z inflating: build/lib/torch/ao/quantization/fx/qconfig_mapping_utils.py 2024-08-08T20:48:47.4539090Z inflating: build/lib/torch/ao/quantization/fx/quantize_handler.py 2024-08-08T20:48:47.4540086Z inflating: build/lib/torch/ao/quantization/fx/tracer.py 2024-08-08T20:48:47.4544523Z inflating: build/lib/torch/ao/quantization/fx/utils.py 2024-08-08T20:48:47.4545298Z creating: build/lib/torch/ao/quantization/fx/_model_report/ 2024-08-08T20:48:47.4546216Z extracting: build/lib/torch/ao/quantization/fx/_model_report/__init__.py 2024-08-08T20:48:47.4552845Z inflating: build/lib/torch/ao/quantization/fx/_model_report/detector.py 2024-08-08T20:48:47.4556314Z inflating: build/lib/torch/ao/quantization/fx/_model_report/model_report.py 2024-08-08T20:48:47.4558129Z inflating: build/lib/torch/ao/quantization/fx/_model_report/model_report_observer.py 2024-08-08T20:48:47.4562362Z inflating: build/lib/torch/ao/quantization/fx/_model_report/model_report_visualizer.py 2024-08-08T20:48:47.4563339Z creating: build/lib/torch/ao/quantization/pt2e/ 2024-08-08T20:48:47.4564065Z extracting: build/lib/torch/ao/quantization/pt2e/__init__.py 2024-08-08T20:48:47.4565021Z inflating: build/lib/torch/ao/quantization/pt2e/_numeric_debugger.py 2024-08-08T20:48:47.4566029Z inflating: build/lib/torch/ao/quantization/pt2e/duplicate_dq_pass.py 2024-08-08T20:48:47.4567326Z inflating: build/lib/torch/ao/quantization/pt2e/export_utils.py 2024-08-08T20:48:47.4568490Z inflating: build/lib/torch/ao/quantization/pt2e/graph_utils.py 2024-08-08T20:48:47.4570199Z inflating: build/lib/torch/ao/quantization/pt2e/port_metadata_pass.py 2024-08-08T20:48:47.4573349Z inflating: build/lib/torch/ao/quantization/pt2e/prepare.py 2024-08-08T20:48:47.4577358Z inflating: build/lib/torch/ao/quantization/pt2e/qat_utils.py 2024-08-08T20:48:47.4580273Z inflating: build/lib/torch/ao/quantization/pt2e/utils.py 2024-08-08T20:48:47.4581102Z creating: build/lib/torch/ao/quantization/pt2e/representation/ 2024-08-08T20:48:47.4582098Z inflating: build/lib/torch/ao/quantization/pt2e/representation/__init__.py 2024-08-08T20:48:47.4584636Z inflating: build/lib/torch/ao/quantization/pt2e/representation/rewrite.py 2024-08-08T20:48:47.4585745Z creating: build/lib/torch/ao/quantization/quantizer/ 2024-08-08T20:48:47.4586581Z inflating: build/lib/torch/ao/quantization/quantizer/__init__.py 2024-08-08T20:48:47.4587622Z inflating: build/lib/torch/ao/quantization/quantizer/composable_quantizer.py 2024-08-08T20:48:47.4588751Z inflating: build/lib/torch/ao/quantization/quantizer/embedding_quantizer.py 2024-08-08T20:48:47.4589830Z inflating: build/lib/torch/ao/quantization/quantizer/quantizer.py 2024-08-08T20:48:47.4590748Z inflating: build/lib/torch/ao/quantization/quantizer/utils.py 2024-08-08T20:48:47.4596584Z inflating: build/lib/torch/ao/quantization/quantizer/x86_inductor_quantizer.py 2024-08-08T20:48:47.4599175Z inflating: build/lib/torch/ao/quantization/quantizer/xnnpack_quantizer.py 2024-08-08T20:48:47.4603295Z inflating: build/lib/torch/ao/quantization/quantizer/xnnpack_quantizer_utils.py 2024-08-08T20:48:47.4604161Z creating: build/lib/torch/autograd/ 2024-08-08T20:48:47.4606538Z inflating: build/lib/torch/autograd/__init__.py 2024-08-08T20:48:47.4607637Z inflating: build/lib/torch/autograd/anomaly_mode.py 2024-08-08T20:48:47.4609118Z inflating: build/lib/torch/autograd/forward_ad.py 2024-08-08T20:48:47.4613222Z inflating: build/lib/torch/autograd/function.py 2024-08-08T20:48:47.4617918Z inflating: build/lib/torch/autograd/functional.py 2024-08-08T20:48:47.4619724Z inflating: build/lib/torch/autograd/grad_mode.py 2024-08-08T20:48:47.4628264Z inflating: build/lib/torch/autograd/gradcheck.py 2024-08-08T20:48:47.4631667Z inflating: build/lib/torch/autograd/graph.py 2024-08-08T20:48:47.4636500Z inflating: build/lib/torch/autograd/profiler.py 2024-08-08T20:48:47.4638637Z inflating: build/lib/torch/autograd/profiler_legacy.py 2024-08-08T20:48:47.4643127Z inflating: build/lib/torch/autograd/profiler_util.py 2024-08-08T20:48:47.4643819Z inflating: build/lib/torch/autograd/variable.py 2024-08-08T20:48:47.4644487Z creating: build/lib/torch/autograd/_functions/ 2024-08-08T20:48:47.4645244Z extracting: build/lib/torch/autograd/_functions/__init__.py 2024-08-08T20:48:47.4646217Z inflating: build/lib/torch/autograd/_functions/tensor.py 2024-08-08T20:48:47.4647130Z inflating: build/lib/torch/autograd/_functions/utils.py 2024-08-08T20:48:47.4647796Z creating: build/lib/torch/backends/ 2024-08-08T20:48:47.4648528Z inflating: build/lib/torch/backends/__init__.py 2024-08-08T20:48:47.4649168Z creating: build/lib/torch/backends/_coreml/ 2024-08-08T20:48:47.4649859Z extracting: build/lib/torch/backends/_coreml/__init__.py 2024-08-08T20:48:47.4650854Z inflating: build/lib/torch/backends/_coreml/preprocess.py 2024-08-08T20:48:47.4651592Z creating: build/lib/torch/backends/_nnapi/ 2024-08-08T20:48:47.4652258Z extracting: build/lib/torch/backends/_nnapi/__init__.py 2024-08-08T20:48:47.4653257Z inflating: build/lib/torch/backends/_nnapi/prepare.py 2024-08-08T20:48:47.4661317Z inflating: build/lib/torch/backends/_nnapi/serializer.py 2024-08-08T20:48:47.4662012Z creating: build/lib/torch/backends/cpu/ 2024-08-08T20:48:47.4662654Z inflating: build/lib/torch/backends/cpu/__init__.py 2024-08-08T20:48:47.4663288Z creating: build/lib/torch/backends/cuda/ 2024-08-08T20:48:47.4664954Z inflating: build/lib/torch/backends/cuda/__init__.py 2024-08-08T20:48:47.4665597Z creating: build/lib/torch/backends/cudnn/ 2024-08-08T20:48:47.4666620Z inflating: build/lib/torch/backends/cudnn/__init__.py 2024-08-08T20:48:47.4667637Z inflating: build/lib/torch/backends/cudnn/rnn.py 2024-08-08T20:48:47.4668257Z creating: build/lib/torch/backends/mha/ 2024-08-08T20:48:47.4669073Z inflating: build/lib/torch/backends/mha/__init__.py 2024-08-08T20:48:47.4669705Z creating: build/lib/torch/backends/mkl/ 2024-08-08T20:48:47.4670996Z inflating: build/lib/torch/backends/mkl/__init__.py 2024-08-08T20:48:47.4671663Z creating: build/lib/torch/backends/mkldnn/ 2024-08-08T20:48:47.4672560Z inflating: build/lib/torch/backends/mkldnn/__init__.py 2024-08-08T20:48:47.4673417Z creating: build/lib/torch/backends/mps/ 2024-08-08T20:48:47.4674496Z inflating: build/lib/torch/backends/mps/__init__.py 2024-08-08T20:48:47.4675248Z creating: build/lib/torch/backends/nnpack/ 2024-08-08T20:48:47.4675929Z inflating: build/lib/torch/backends/nnpack/__init__.py 2024-08-08T20:48:47.4676622Z creating: build/lib/torch/backends/openmp/ 2024-08-08T20:48:47.4677309Z inflating: build/lib/torch/backends/openmp/__init__.py 2024-08-08T20:48:47.4677991Z creating: build/lib/torch/backends/opt_einsum/ 2024-08-08T20:48:47.4678733Z inflating: build/lib/torch/backends/opt_einsum/__init__.py 2024-08-08T20:48:47.4679469Z creating: build/lib/torch/backends/quantized/ 2024-08-08T20:48:47.4680417Z inflating: build/lib/torch/backends/quantized/__init__.py 2024-08-08T20:48:47.4681099Z creating: build/lib/torch/backends/xeon/ 2024-08-08T20:48:47.4681768Z extracting: build/lib/torch/backends/xeon/__init__.py 2024-08-08T20:48:47.4685839Z inflating: build/lib/torch/backends/xeon/run_cpu.py 2024-08-08T20:48:47.4686546Z creating: build/lib/torch/backends/xnnpack/ 2024-08-08T20:48:47.4687225Z inflating: build/lib/torch/backends/xnnpack/__init__.py 2024-08-08T20:48:47.4688382Z creating: build/lib/torch/compiler/ 2024-08-08T20:48:47.4689044Z inflating: build/lib/torch/compiler/__init__.py 2024-08-08T20:48:47.4689693Z creating: build/lib/torch/contrib/ 2024-08-08T20:48:47.4690288Z extracting: build/lib/torch/contrib/__init__.py 2024-08-08T20:48:47.4691588Z inflating: build/lib/torch/contrib/_tensorboard_vis.py 2024-08-08T20:48:47.4692220Z creating: build/lib/torch/cpu/ 2024-08-08T20:48:47.4693105Z inflating: build/lib/torch/cpu/__init__.py 2024-08-08T20:48:47.4693691Z creating: build/lib/torch/cpu/amp/ 2024-08-08T20:48:47.4694282Z inflating: build/lib/torch/cpu/amp/__init__.py 2024-08-08T20:48:47.4695242Z inflating: build/lib/torch/cpu/amp/autocast_mode.py 2024-08-08T20:48:47.4696137Z inflating: build/lib/torch/cpu/amp/grad_scaler.py 2024-08-08T20:48:47.4696774Z creating: build/lib/torch/cuda/ 2024-08-08T20:48:47.4702252Z inflating: build/lib/torch/cuda/__init__.py 2024-08-08T20:48:47.4702913Z inflating: build/lib/torch/cuda/_gpu_trace.py 2024-08-08T20:48:47.4706319Z inflating: build/lib/torch/cuda/_memory_viz.py 2024-08-08T20:48:47.4708749Z inflating: build/lib/torch/cuda/_sanitizer.py 2024-08-08T20:48:47.4709380Z inflating: build/lib/torch/cuda/_utils.py 2024-08-08T20:48:47.4710196Z inflating: build/lib/torch/cuda/comm.py 2024-08-08T20:48:47.4710810Z extracting: build/lib/torch/cuda/error.py 2024-08-08T20:48:47.4713752Z inflating: build/lib/torch/cuda/graphs.py 2024-08-08T20:48:47.4714954Z inflating: build/lib/torch/cuda/jiterator.py 2024-08-08T20:48:47.4719238Z inflating: build/lib/torch/cuda/memory.py 2024-08-08T20:48:47.4720536Z inflating: build/lib/torch/cuda/nccl.py 2024-08-08T20:48:47.4721601Z inflating: build/lib/torch/cuda/nvtx.py 2024-08-08T20:48:47.4722709Z inflating: build/lib/torch/cuda/profiler.py 2024-08-08T20:48:47.4723986Z inflating: build/lib/torch/cuda/random.py 2024-08-08T20:48:47.4724929Z inflating: build/lib/torch/cuda/sparse.py 2024-08-08T20:48:47.4726824Z inflating: build/lib/torch/cuda/streams.py 2024-08-08T20:48:47.4728607Z inflating: build/lib/torch/cuda/tunable.py 2024-08-08T20:48:47.4729206Z creating: build/lib/torch/cuda/amp/ 2024-08-08T20:48:47.4729810Z inflating: build/lib/torch/cuda/amp/__init__.py 2024-08-08T20:48:47.4730666Z inflating: build/lib/torch/cuda/amp/autocast_mode.py 2024-08-08T20:48:47.4731356Z inflating: build/lib/torch/cuda/amp/common.py 2024-08-08T20:48:47.4732195Z inflating: build/lib/torch/cuda/amp/grad_scaler.py 2024-08-08T20:48:47.4732833Z creating: build/lib/torch/distributed/ 2024-08-08T20:48:47.4733898Z inflating: build/lib/torch/distributed/__init__.py 2024-08-08T20:48:47.4734846Z inflating: build/lib/torch/distributed/_checkpointable.py 2024-08-08T20:48:47.4736010Z inflating: build/lib/torch/distributed/_composable_state.py 2024-08-08T20:48:47.4740288Z inflating: build/lib/torch/distributed/_functional_collectives.py 2024-08-08T20:48:47.4741287Z inflating: build/lib/torch/distributed/_functional_collectives_impl.py 2024-08-08T20:48:47.4744274Z inflating: build/lib/torch/distributed/_state_dict_utils.py 2024-08-08T20:48:47.4745503Z inflating: build/lib/torch/distributed/argparse_util.py 2024-08-08T20:48:47.4746631Z inflating: build/lib/torch/distributed/c10d_logger.py 2024-08-08T20:48:47.4748521Z inflating: build/lib/torch/distributed/collective_utils.py 2024-08-08T20:48:47.4749313Z inflating: build/lib/torch/distributed/constants.py 2024-08-08T20:48:47.4752924Z inflating: build/lib/torch/distributed/device_mesh.py 2024-08-08T20:48:47.4770030Z inflating: build/lib/torch/distributed/distributed_c10d.py 2024-08-08T20:48:47.4771375Z inflating: build/lib/torch/distributed/launch.py 2024-08-08T20:48:47.4772135Z inflating: build/lib/torch/distributed/logging_handlers.py 2024-08-08T20:48:47.4773290Z inflating: build/lib/torch/distributed/remote_device.py 2024-08-08T20:48:47.4775320Z inflating: build/lib/torch/distributed/rendezvous.py 2024-08-08T20:48:47.4779396Z inflating: build/lib/torch/distributed/run.py 2024-08-08T20:48:47.4781133Z inflating: build/lib/torch/distributed/utils.py 2024-08-08T20:48:47.4781835Z creating: build/lib/torch/distributed/_composable/ 2024-08-08T20:48:47.4783015Z inflating: build/lib/torch/distributed/_composable/__init__.py 2024-08-08T20:48:47.4784008Z inflating: build/lib/torch/distributed/_composable/checkpoint_activation.py 2024-08-08T20:48:47.4785555Z inflating: build/lib/torch/distributed/_composable/contract.py 2024-08-08T20:48:47.4786700Z inflating: build/lib/torch/distributed/_composable/fully_shard.py 2024-08-08T20:48:47.4788442Z inflating: build/lib/torch/distributed/_composable/replicate.py 2024-08-08T20:48:47.4789271Z creating: build/lib/torch/distributed/_composable/fsdp/ 2024-08-08T20:48:47.4790187Z inflating: build/lib/torch/distributed/_composable/fsdp/__init__.py 2024-08-08T20:48:47.4791360Z inflating: build/lib/torch/distributed/_composable/fsdp/_fsdp_api.py 2024-08-08T20:48:47.4793962Z inflating: build/lib/torch/distributed/_composable/fsdp/_fsdp_collectives.py 2024-08-08T20:48:47.4795289Z inflating: build/lib/torch/distributed/_composable/fsdp/_fsdp_common.py 2024-08-08T20:48:47.4796478Z inflating: build/lib/torch/distributed/_composable/fsdp/_fsdp_init.py 2024-08-08T20:48:47.4800573Z inflating: build/lib/torch/distributed/_composable/fsdp/_fsdp_param.py 2024-08-08T20:48:47.4803884Z inflating: build/lib/torch/distributed/_composable/fsdp/_fsdp_param_group.py 2024-08-08T20:48:47.4806180Z inflating: build/lib/torch/distributed/_composable/fsdp/_fsdp_state.py 2024-08-08T20:48:47.4809109Z inflating: build/lib/torch/distributed/_composable/fsdp/fully_shard.py 2024-08-08T20:48:47.4809919Z creating: build/lib/torch/distributed/_shard/ 2024-08-08T20:48:47.4810672Z inflating: build/lib/torch/distributed/_shard/__init__.py 2024-08-08T20:48:47.4811356Z inflating: build/lib/torch/distributed/_shard/_utils.py 2024-08-08T20:48:47.4812764Z inflating: build/lib/torch/distributed/_shard/api.py 2024-08-08T20:48:47.4813844Z inflating: build/lib/torch/distributed/_shard/common_op_utils.py 2024-08-08T20:48:47.4814843Z inflating: build/lib/torch/distributed/_shard/metadata.py 2024-08-08T20:48:47.4815843Z inflating: build/lib/torch/distributed/_shard/op_registry_utils.py 2024-08-08T20:48:47.4816716Z inflating: build/lib/torch/distributed/_shard/sharder.py 2024-08-08T20:48:47.4817502Z creating: build/lib/torch/distributed/_shard/checkpoint/ 2024-08-08T20:48:47.4818371Z inflating: build/lib/torch/distributed/_shard/checkpoint/__init__.py 2024-08-08T20:48:47.4819220Z creating: build/lib/torch/distributed/_shard/sharded_optim/ 2024-08-08T20:48:47.4820134Z inflating: build/lib/torch/distributed/_shard/sharded_optim/__init__.py 2024-08-08T20:48:47.4821379Z inflating: build/lib/torch/distributed/_shard/sharded_optim/api.py 2024-08-08T20:48:47.4822192Z creating: build/lib/torch/distributed/_shard/sharded_tensor/ 2024-08-08T20:48:47.4823455Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/__init__.py 2024-08-08T20:48:47.4828678Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/api.py 2024-08-08T20:48:47.4829598Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/logger.py 2024-08-08T20:48:47.4830668Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/logging_handlers.py 2024-08-08T20:48:47.4831743Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/metadata.py 2024-08-08T20:48:47.4832985Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/reshard.py 2024-08-08T20:48:47.4834232Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/shard.py 2024-08-08T20:48:47.4835551Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/utils.py 2024-08-08T20:48:47.4836515Z creating: build/lib/torch/distributed/_shard/sharded_tensor/_ops/ 2024-08-08T20:48:47.4837632Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/_ops/__init__.py 2024-08-08T20:48:47.4838741Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/_ops/_common.py 2024-08-08T20:48:47.4839849Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/_ops/binary_cmp.py 2024-08-08T20:48:47.4840930Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/_ops/init.py 2024-08-08T20:48:47.4842001Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/_ops/misc_ops.py 2024-08-08T20:48:47.4843279Z inflating: build/lib/torch/distributed/_shard/sharded_tensor/_ops/tensor_ops.py 2024-08-08T20:48:47.4844228Z creating: build/lib/torch/distributed/_shard/sharding_plan/ 2024-08-08T20:48:47.4845140Z inflating: build/lib/torch/distributed/_shard/sharding_plan/__init__.py 2024-08-08T20:48:47.4846113Z inflating: build/lib/torch/distributed/_shard/sharding_plan/api.py 2024-08-08T20:48:47.4846987Z creating: build/lib/torch/distributed/_shard/sharding_spec/ 2024-08-08T20:48:47.4847906Z inflating: build/lib/torch/distributed/_shard/sharding_spec/__init__.py 2024-08-08T20:48:47.4848948Z inflating: build/lib/torch/distributed/_shard/sharding_spec/_internals.py 2024-08-08T20:48:47.4850259Z inflating: build/lib/torch/distributed/_shard/sharding_spec/api.py 2024-08-08T20:48:47.4852037Z inflating: build/lib/torch/distributed/_shard/sharding_spec/chunk_sharding_spec.py 2024-08-08T20:48:47.4853175Z creating: build/lib/torch/distributed/_shard/sharding_spec/chunk_sharding_spec_ops/ 2024-08-08T20:48:47.4854425Z extracting: build/lib/torch/distributed/_shard/sharding_spec/chunk_sharding_spec_ops/__init__.py 2024-08-08T20:48:47.4855751Z inflating: build/lib/torch/distributed/_shard/sharding_spec/chunk_sharding_spec_ops/_common.py 2024-08-08T20:48:47.4857223Z inflating: build/lib/torch/distributed/_shard/sharding_spec/chunk_sharding_spec_ops/embedding.py 2024-08-08T20:48:47.4860834Z inflating: build/lib/torch/distributed/_shard/sharding_spec/chunk_sharding_spec_ops/embedding_bag.py 2024-08-08T20:48:47.4861923Z creating: build/lib/torch/distributed/_sharded_tensor/ 2024-08-08T20:48:47.4862751Z inflating: build/lib/torch/distributed/_sharded_tensor/__init__.py 2024-08-08T20:48:47.4863593Z creating: build/lib/torch/distributed/_sharding_spec/ 2024-08-08T20:48:47.4864412Z inflating: build/lib/torch/distributed/_sharding_spec/__init__.py 2024-08-08T20:48:47.4865235Z creating: build/lib/torch/distributed/_symmetric_memory/ 2024-08-08T20:48:47.4866909Z inflating: build/lib/torch/distributed/_symmetric_memory/__init__.py 2024-08-08T20:48:47.4867729Z creating: build/lib/torch/distributed/_tensor/ 2024-08-08T20:48:47.4869002Z inflating: build/lib/torch/distributed/_tensor/__init__.py 2024-08-08T20:48:47.4871364Z inflating: build/lib/torch/distributed/_tensor/_collective_utils.py 2024-08-08T20:48:47.4873610Z inflating: build/lib/torch/distributed/_tensor/_dispatch.py 2024-08-08T20:48:47.4875883Z inflating: build/lib/torch/distributed/_tensor/_op_schema.py 2024-08-08T20:48:47.4877926Z inflating: build/lib/torch/distributed/_tensor/_redistribute.py 2024-08-08T20:48:47.4881019Z inflating: build/lib/torch/distributed/_tensor/_sharding_prop.py 2024-08-08T20:48:47.4882581Z inflating: build/lib/torch/distributed/_tensor/_shards_wrapper.py 2024-08-08T20:48:47.4884049Z inflating: build/lib/torch/distributed/_tensor/_tp_conv.py 2024-08-08T20:48:47.4886137Z inflating: build/lib/torch/distributed/_tensor/_utils.py 2024-08-08T20:48:47.4890058Z inflating: build/lib/torch/distributed/_tensor/api.py 2024-08-08T20:48:47.4891281Z inflating: build/lib/torch/distributed/_tensor/device_mesh.py 2024-08-08T20:48:47.4894585Z inflating: build/lib/torch/distributed/_tensor/placement_types.py 2024-08-08T20:48:47.4896888Z inflating: build/lib/torch/distributed/_tensor/random.py 2024-08-08T20:48:47.4897644Z creating: build/lib/torch/distributed/_tensor/debug/ 2024-08-08T20:48:47.4898462Z inflating: build/lib/torch/distributed/_tensor/debug/__init__.py 2024-08-08T20:48:47.4899561Z inflating: build/lib/torch/distributed/_tensor/debug/_op_coverage.py 2024-08-08T20:48:47.4902522Z inflating: build/lib/torch/distributed/_tensor/debug/comm_mode.py 2024-08-08T20:48:47.4903870Z inflating: build/lib/torch/distributed/_tensor/debug/visualize_sharding.py 2024-08-08T20:48:47.4904952Z creating: build/lib/torch/distributed/_tensor/experimental/ 2024-08-08T20:48:47.4905860Z inflating: build/lib/torch/distributed/_tensor/experimental/__init__.py 2024-08-08T20:48:47.4908656Z inflating: build/lib/torch/distributed/_tensor/experimental/attention.py 2024-08-08T20:48:47.4910168Z inflating: build/lib/torch/distributed/_tensor/experimental/local_map.py 2024-08-08T20:48:47.4911584Z inflating: build/lib/torch/distributed/_tensor/experimental/register_sharding.py 2024-08-08T20:48:47.4914335Z inflating: build/lib/torch/distributed/_tensor/experimental/tp_transform.py 2024-08-08T20:48:47.4915238Z creating: build/lib/torch/distributed/_tensor/ops/ 2024-08-08T20:48:47.4916026Z inflating: build/lib/torch/distributed/_tensor/ops/__init__.py 2024-08-08T20:48:47.4917316Z inflating: build/lib/torch/distributed/_tensor/ops/_common_rules.py 2024-08-08T20:48:47.4918380Z inflating: build/lib/torch/distributed/_tensor/ops/_conv_ops.py 2024-08-08T20:48:47.4919851Z inflating: build/lib/torch/distributed/_tensor/ops/_einsum_strategy.py 2024-08-08T20:48:47.4921548Z inflating: build/lib/torch/distributed/_tensor/ops/_embedding_ops.py 2024-08-08T20:48:47.4922540Z inflating: build/lib/torch/distributed/_tensor/ops/_experimental_ops.py 2024-08-08T20:48:47.4926494Z inflating: build/lib/torch/distributed/_tensor/ops/_math_ops.py 2024-08-08T20:48:47.4928513Z inflating: build/lib/torch/distributed/_tensor/ops/_matrix_ops.py 2024-08-08T20:48:47.4931180Z inflating: build/lib/torch/distributed/_tensor/ops/_pointwise_ops.py 2024-08-08T20:48:47.4932115Z inflating: build/lib/torch/distributed/_tensor/ops/_random_ops.py 2024-08-08T20:48:47.4935430Z inflating: build/lib/torch/distributed/_tensor/ops/_tensor_ops.py 2024-08-08T20:48:47.4938289Z inflating: build/lib/torch/distributed/_tensor/ops/_view_ops.py 2024-08-08T20:48:47.4940351Z inflating: build/lib/torch/distributed/_tensor/ops/utils.py 2024-08-08T20:48:47.4941094Z creating: build/lib/torch/distributed/_tools/ 2024-08-08T20:48:47.4941810Z inflating: build/lib/torch/distributed/_tools/__init__.py 2024-08-08T20:48:47.4944580Z inflating: build/lib/torch/distributed/_tools/fsdp2_mem_tracker.py 2024-08-08T20:48:47.4948790Z inflating: build/lib/torch/distributed/_tools/mem_tracker.py 2024-08-08T20:48:47.4950432Z inflating: build/lib/torch/distributed/_tools/memory_tracker.py 2024-08-08T20:48:47.4952109Z inflating: build/lib/torch/distributed/_tools/mod_tracker.py 2024-08-08T20:48:47.4953032Z creating: build/lib/torch/distributed/algorithms/ 2024-08-08T20:48:47.4953799Z inflating: build/lib/torch/distributed/algorithms/__init__.py 2024-08-08T20:48:47.4955351Z inflating: build/lib/torch/distributed/algorithms/join.py 2024-08-08T20:48:47.4956215Z creating: build/lib/torch/distributed/algorithms/_checkpoint/ 2024-08-08T20:48:47.4957215Z extracting: build/lib/torch/distributed/algorithms/_checkpoint/__init__.py 2024-08-08T20:48:47.4958922Z inflating: build/lib/torch/distributed/algorithms/_checkpoint/checkpoint_wrapper.py 2024-08-08T20:48:47.4960009Z creating: build/lib/torch/distributed/algorithms/_comm_hooks/ 2024-08-08T20:48:47.4960965Z inflating: build/lib/torch/distributed/algorithms/_comm_hooks/__init__.py 2024-08-08T20:48:47.4962052Z inflating: build/lib/torch/distributed/algorithms/_comm_hooks/default_hooks.py 2024-08-08T20:48:47.4963106Z creating: build/lib/torch/distributed/algorithms/_optimizer_overlap/ 2024-08-08T20:48:47.4964188Z inflating: build/lib/torch/distributed/algorithms/_optimizer_overlap/__init__.py 2024-08-08T20:48:47.4965399Z inflating: build/lib/torch/distributed/algorithms/_optimizer_overlap/optimizer_overlap.py 2024-08-08T20:48:47.4966654Z creating: build/lib/torch/distributed/algorithms/_quantization/ 2024-08-08T20:48:47.4967686Z extracting: build/lib/torch/distributed/algorithms/_quantization/__init__.py 2024-08-08T20:48:47.4968825Z inflating: build/lib/torch/distributed/algorithms/_quantization/quantization.py 2024-08-08T20:48:47.4969846Z creating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/ 2024-08-08T20:48:47.4970872Z inflating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/__init__.py 2024-08-08T20:48:47.4972018Z inflating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/ddp_zero_hook.py 2024-08-08T20:48:47.4973203Z inflating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/debugging_hooks.py 2024-08-08T20:48:47.4974399Z inflating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/default_hooks.py 2024-08-08T20:48:47.4975659Z inflating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/mixed_precision_hooks.py 2024-08-08T20:48:47.4976969Z inflating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/optimizer_overlap_hooks.py 2024-08-08T20:48:47.4978234Z inflating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/post_localSGD_hook.py 2024-08-08T20:48:47.4981481Z inflating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/powerSGD_hook.py 2024-08-08T20:48:47.4982847Z inflating: build/lib/torch/distributed/algorithms/ddp_comm_hooks/quantization_hooks.py 2024-08-08T20:48:47.4983947Z creating: build/lib/torch/distributed/algorithms/model_averaging/ 2024-08-08T20:48:47.4984962Z extracting: build/lib/torch/distributed/algorithms/model_averaging/__init__.py 2024-08-08T20:48:47.4986114Z inflating: build/lib/torch/distributed/algorithms/model_averaging/averagers.py 2024-08-08T20:48:47.4987416Z inflating: build/lib/torch/distributed/algorithms/model_averaging/hierarchical_model_averager.py 2024-08-08T20:48:47.4988674Z inflating: build/lib/torch/distributed/algorithms/model_averaging/utils.py 2024-08-08T20:48:47.4988917Z creating: build/lib/torch/distributed/autograd/ 2024-08-08T20:48:47.4989487Z inflating: build/lib/torch/distributed/autograd/__init__.py 2024-08-08T20:48:47.4990308Z creating: build/lib/torch/distributed/checkpoint/ 2024-08-08T20:48:47.4991128Z inflating: build/lib/torch/distributed/checkpoint/__init__.py 2024-08-08T20:48:47.4992394Z inflating: build/lib/torch/distributed/checkpoint/_checkpointer.py 2024-08-08T20:48:47.4993416Z inflating: build/lib/torch/distributed/checkpoint/_dedup_save_plans.py 2024-08-08T20:48:47.4994365Z inflating: build/lib/torch/distributed/checkpoint/_dedup_tensors.py 2024-08-08T20:48:47.4995669Z inflating: build/lib/torch/distributed/checkpoint/_fsspec_filesystem.py 2024-08-08T20:48:47.4996515Z inflating: build/lib/torch/distributed/checkpoint/_nested_dict.py 2024-08-08T20:48:47.4997891Z inflating: build/lib/torch/distributed/checkpoint/_sharded_tensor_utils.py 2024-08-08T20:48:47.4998929Z inflating: build/lib/torch/distributed/checkpoint/_storage_utils.py 2024-08-08T20:48:47.5000307Z inflating: build/lib/torch/distributed/checkpoint/_traverse.py 2024-08-08T20:48:47.5001181Z inflating: build/lib/torch/distributed/checkpoint/api.py 2024-08-08T20:48:47.5003991Z inflating: build/lib/torch/distributed/checkpoint/default_planner.py 2024-08-08T20:48:47.5006924Z inflating: build/lib/torch/distributed/checkpoint/filesystem.py 2024-08-08T20:48:47.5008754Z inflating: build/lib/torch/distributed/checkpoint/format_utils.py 2024-08-08T20:48:47.5009381Z inflating: build/lib/torch/distributed/checkpoint/logger.py 2024-08-08T20:48:47.5010482Z inflating: build/lib/torch/distributed/checkpoint/logging_handlers.py 2024-08-08T20:48:47.5011777Z inflating: build/lib/torch/distributed/checkpoint/metadata.py 2024-08-08T20:48:47.5013922Z inflating: build/lib/torch/distributed/checkpoint/optimizer.py 2024-08-08T20:48:47.5016009Z inflating: build/lib/torch/distributed/checkpoint/planner.py 2024-08-08T20:48:47.5018112Z inflating: build/lib/torch/distributed/checkpoint/planner_helpers.py 2024-08-08T20:48:47.5018700Z inflating: build/lib/torch/distributed/checkpoint/resharding.py 2024-08-08T20:48:47.5019987Z inflating: build/lib/torch/distributed/checkpoint/staging.py 2024-08-08T20:48:47.5025126Z inflating: build/lib/torch/distributed/checkpoint/state_dict.py 2024-08-08T20:48:47.5027003Z inflating: build/lib/torch/distributed/checkpoint/state_dict_loader.py 2024-08-08T20:48:47.5029144Z inflating: build/lib/torch/distributed/checkpoint/state_dict_saver.py 2024-08-08T20:48:47.5029889Z inflating: build/lib/torch/distributed/checkpoint/stateful.py 2024-08-08T20:48:47.5031727Z inflating: build/lib/torch/distributed/checkpoint/storage.py 2024-08-08T20:48:47.5033892Z inflating: build/lib/torch/distributed/checkpoint/utils.py 2024-08-08T20:48:47.5034176Z creating: build/lib/torch/distributed/elastic/ 2024-08-08T20:48:47.5035263Z inflating: build/lib/torch/distributed/elastic/__init__.py 2024-08-08T20:48:47.5036060Z inflating: build/lib/torch/distributed/elastic/control_plane.py 2024-08-08T20:48:47.5036521Z creating: build/lib/torch/distributed/elastic/agent/ 2024-08-08T20:48:47.5037139Z extracting: build/lib/torch/distributed/elastic/agent/__init__.py 2024-08-08T20:48:47.5037614Z creating: build/lib/torch/distributed/elastic/agent/server/ 2024-08-08T20:48:47.5038810Z inflating: build/lib/torch/distributed/elastic/agent/server/__init__.py 2024-08-08T20:48:47.5042906Z inflating: build/lib/torch/distributed/elastic/agent/server/api.py 2024-08-08T20:48:47.5043632Z inflating: build/lib/torch/distributed/elastic/agent/server/health_check_server.py 2024-08-08T20:48:47.5046090Z inflating: build/lib/torch/distributed/elastic/agent/server/local_elastic_agent.py 2024-08-08T20:48:47.5046381Z creating: build/lib/torch/distributed/elastic/events/ 2024-08-08T20:48:47.5047469Z inflating: build/lib/torch/distributed/elastic/events/__init__.py 2024-08-08T20:48:47.5048852Z inflating: build/lib/torch/distributed/elastic/events/api.py 2024-08-08T20:48:47.5049843Z inflating: build/lib/torch/distributed/elastic/events/handlers.py 2024-08-08T20:48:47.5050317Z creating: build/lib/torch/distributed/elastic/metrics/ 2024-08-08T20:48:47.5051735Z inflating: build/lib/torch/distributed/elastic/metrics/__init__.py 2024-08-08T20:48:47.5053144Z inflating: build/lib/torch/distributed/elastic/metrics/api.py 2024-08-08T20:48:47.5053639Z creating: build/lib/torch/distributed/elastic/multiprocessing/ 2024-08-08T20:48:47.5055477Z inflating: build/lib/torch/distributed/elastic/multiprocessing/__init__.py 2024-08-08T20:48:47.5059233Z inflating: build/lib/torch/distributed/elastic/multiprocessing/api.py 2024-08-08T20:48:47.5060385Z inflating: build/lib/torch/distributed/elastic/multiprocessing/redirects.py 2024-08-08T20:48:47.5061906Z inflating: build/lib/torch/distributed/elastic/multiprocessing/tail_log.py 2024-08-08T20:48:47.5062363Z creating: build/lib/torch/distributed/elastic/multiprocessing/errors/ 2024-08-08T20:48:47.5064595Z inflating: build/lib/torch/distributed/elastic/multiprocessing/errors/__init__.py 2024-08-08T20:48:47.5066224Z inflating: build/lib/torch/distributed/elastic/multiprocessing/errors/error_handler.py 2024-08-08T20:48:47.5067093Z inflating: build/lib/torch/distributed/elastic/multiprocessing/errors/handlers.py 2024-08-08T20:48:47.5067925Z creating: build/lib/torch/distributed/elastic/multiprocessing/subprocess_handler/ 2024-08-08T20:48:47.5068843Z inflating: build/lib/torch/distributed/elastic/multiprocessing/subprocess_handler/__init__.py 2024-08-08T20:48:47.5069763Z inflating: build/lib/torch/distributed/elastic/multiprocessing/subprocess_handler/handlers.py 2024-08-08T20:48:47.5070689Z inflating: build/lib/torch/distributed/elastic/multiprocessing/subprocess_handler/subprocess_handler.py 2024-08-08T20:48:47.5071014Z creating: build/lib/torch/distributed/elastic/rendezvous/ 2024-08-08T20:48:47.5072439Z inflating: build/lib/torch/distributed/elastic/rendezvous/__init__.py 2024-08-08T20:48:47.5074341Z inflating: build/lib/torch/distributed/elastic/rendezvous/api.py 2024-08-08T20:48:47.5076275Z inflating: build/lib/torch/distributed/elastic/rendezvous/c10d_rendezvous_backend.py 2024-08-08T20:48:47.5080776Z inflating: build/lib/torch/distributed/elastic/rendezvous/dynamic_rendezvous.py 2024-08-08T20:48:47.5085779Z inflating: build/lib/torch/distributed/elastic/rendezvous/etcd_rendezvous.py 2024-08-08T20:48:47.5087369Z inflating: build/lib/torch/distributed/elastic/rendezvous/etcd_rendezvous_backend.py 2024-08-08T20:48:47.5089258Z inflating: build/lib/torch/distributed/elastic/rendezvous/etcd_server.py 2024-08-08T20:48:47.5090855Z inflating: build/lib/torch/distributed/elastic/rendezvous/etcd_store.py 2024-08-08T20:48:47.5091923Z inflating: build/lib/torch/distributed/elastic/rendezvous/registry.py 2024-08-08T20:48:47.5093262Z inflating: build/lib/torch/distributed/elastic/rendezvous/static_tcp_rendezvous.py 2024-08-08T20:48:47.5095034Z inflating: build/lib/torch/distributed/elastic/rendezvous/utils.py 2024-08-08T20:48:47.5095426Z creating: build/lib/torch/distributed/elastic/timer/ 2024-08-08T20:48:47.5096192Z inflating: build/lib/torch/distributed/elastic/timer/__init__.py 2024-08-08T20:48:47.5098189Z inflating: build/lib/torch/distributed/elastic/timer/api.py 2024-08-08T20:48:47.5098817Z inflating: build/lib/torch/distributed/elastic/timer/debug_info_logging.py 2024-08-08T20:48:47.5101138Z inflating: build/lib/torch/distributed/elastic/timer/file_based_local_timer.py 2024-08-08T20:48:47.5102263Z inflating: build/lib/torch/distributed/elastic/timer/local_timer.py 2024-08-08T20:48:47.5102756Z creating: build/lib/torch/distributed/elastic/utils/ 2024-08-08T20:48:47.5103442Z inflating: build/lib/torch/distributed/elastic/utils/__init__.py 2024-08-08T20:48:47.5104354Z inflating: build/lib/torch/distributed/elastic/utils/api.py 2024-08-08T20:48:47.5106055Z inflating: build/lib/torch/distributed/elastic/utils/distributed.py 2024-08-08T20:48:47.5106616Z inflating: build/lib/torch/distributed/elastic/utils/log_level.py 2024-08-08T20:48:47.5107765Z inflating: build/lib/torch/distributed/elastic/utils/logging.py 2024-08-08T20:48:47.5108953Z inflating: build/lib/torch/distributed/elastic/utils/store.py 2024-08-08T20:48:47.5109447Z creating: build/lib/torch/distributed/elastic/utils/data/ 2024-08-08T20:48:47.5110376Z inflating: build/lib/torch/distributed/elastic/utils/data/__init__.py 2024-08-08T20:48:47.5111432Z inflating: build/lib/torch/distributed/elastic/utils/data/cycling_iterator.py 2024-08-08T20:48:47.5112837Z inflating: build/lib/torch/distributed/elastic/utils/data/elastic_distributed_sampler.py 2024-08-08T20:48:47.5113479Z creating: build/lib/torch/distributed/fsdp/ 2024-08-08T20:48:47.5113857Z inflating: build/lib/torch/distributed/fsdp/__init__.py 2024-08-08T20:48:47.5116945Z inflating: build/lib/torch/distributed/fsdp/_common_utils.py 2024-08-08T20:48:47.5118425Z inflating: build/lib/torch/distributed/fsdp/_debug_utils.py 2024-08-08T20:48:47.5119498Z inflating: build/lib/torch/distributed/fsdp/_dynamo_utils.py 2024-08-08T20:48:47.5121773Z inflating: build/lib/torch/distributed/fsdp/_exec_order_utils.py 2024-08-08T20:48:47.5132748Z inflating: build/lib/torch/distributed/fsdp/_flat_param.py 2024-08-08T20:48:47.5133986Z inflating: build/lib/torch/distributed/fsdp/_fsdp_extensions.py 2024-08-08T20:48:47.5139010Z inflating: build/lib/torch/distributed/fsdp/_init_utils.py 2024-08-08T20:48:47.5139597Z inflating: build/lib/torch/distributed/fsdp/_limiter_utils.py 2024-08-08T20:48:47.5147942Z inflating: build/lib/torch/distributed/fsdp/_optim_utils.py 2024-08-08T20:48:47.5154502Z inflating: build/lib/torch/distributed/fsdp/_runtime_utils.py 2024-08-08T20:48:47.5155959Z inflating: build/lib/torch/distributed/fsdp/_shard_utils.py 2024-08-08T20:48:47.5160161Z inflating: build/lib/torch/distributed/fsdp/_state_dict_utils.py 2024-08-08T20:48:47.5161724Z inflating: build/lib/torch/distributed/fsdp/_trace_utils.py 2024-08-08T20:48:47.5162837Z inflating: build/lib/torch/distributed/fsdp/_traversal_utils.py 2024-08-08T20:48:47.5164962Z inflating: build/lib/torch/distributed/fsdp/_unshard_param_utils.py 2024-08-08T20:48:47.5166699Z inflating: build/lib/torch/distributed/fsdp/_wrap_utils.py 2024-08-08T20:48:47.5169108Z inflating: build/lib/torch/distributed/fsdp/api.py 2024-08-08T20:48:47.5178369Z inflating: build/lib/torch/distributed/fsdp/fully_sharded_data_parallel.py 2024-08-08T20:48:47.5180782Z inflating: build/lib/torch/distributed/fsdp/sharded_grad_scaler.py 2024-08-08T20:48:47.5183551Z inflating: build/lib/torch/distributed/fsdp/wrap.py 2024-08-08T20:48:47.5183809Z creating: build/lib/torch/distributed/launcher/ 2024-08-08T20:48:47.5184574Z inflating: build/lib/torch/distributed/launcher/__init__.py 2024-08-08T20:48:47.5186761Z inflating: build/lib/torch/distributed/launcher/api.py 2024-08-08T20:48:47.5186979Z creating: build/lib/torch/distributed/nn/ 2024-08-08T20:48:47.5187651Z inflating: build/lib/torch/distributed/nn/__init__.py 2024-08-08T20:48:47.5190032Z inflating: build/lib/torch/distributed/nn/functional.py 2024-08-08T20:48:47.5190268Z creating: build/lib/torch/distributed/nn/api/ 2024-08-08T20:48:47.5191330Z extracting: build/lib/torch/distributed/nn/api/__init__.py 2024-08-08T20:48:47.5195136Z inflating: build/lib/torch/distributed/nn/api/remote_module.py 2024-08-08T20:48:47.5195372Z creating: build/lib/torch/distributed/nn/jit/ 2024-08-08T20:48:47.5195732Z extracting: build/lib/torch/distributed/nn/jit/__init__.py 2024-08-08T20:48:47.5197488Z inflating: build/lib/torch/distributed/nn/jit/instantiator.py 2024-08-08T20:48:47.5197799Z creating: build/lib/torch/distributed/nn/jit/templates/ 2024-08-08T20:48:47.5198396Z extracting: build/lib/torch/distributed/nn/jit/templates/__init__.py 2024-08-08T20:48:47.5200086Z inflating: build/lib/torch/distributed/nn/jit/templates/remote_module_template.py 2024-08-08T20:48:47.5200340Z creating: build/lib/torch/distributed/optim/ 2024-08-08T20:48:47.5201105Z inflating: build/lib/torch/distributed/optim/__init__.py 2024-08-08T20:48:47.5202838Z inflating: build/lib/torch/distributed/optim/apply_optimizer_in_backward.py 2024-08-08T20:48:47.5203589Z inflating: build/lib/torch/distributed/optim/functional_adadelta.py 2024-08-08T20:48:47.5205169Z inflating: build/lib/torch/distributed/optim/functional_adagrad.py 2024-08-08T20:48:47.5206296Z inflating: build/lib/torch/distributed/optim/functional_adam.py 2024-08-08T20:48:47.5207773Z inflating: build/lib/torch/distributed/optim/functional_adamax.py 2024-08-08T20:48:47.5209533Z inflating: build/lib/torch/distributed/optim/functional_adamw.py 2024-08-08T20:48:47.5210577Z inflating: build/lib/torch/distributed/optim/functional_rmsprop.py 2024-08-08T20:48:47.5211773Z inflating: build/lib/torch/distributed/optim/functional_rprop.py 2024-08-08T20:48:47.5213397Z inflating: build/lib/torch/distributed/optim/functional_sgd.py 2024-08-08T20:48:47.5215538Z inflating: build/lib/torch/distributed/optim/named_optimizer.py 2024-08-08T20:48:47.5217839Z inflating: build/lib/torch/distributed/optim/optimizer.py 2024-08-08T20:48:47.5219034Z inflating: build/lib/torch/distributed/optim/post_localSGD_optimizer.py 2024-08-08T20:48:47.5220125Z inflating: build/lib/torch/distributed/optim/utils.py 2024-08-08T20:48:47.5227698Z inflating: build/lib/torch/distributed/optim/zero_redundancy_optimizer.py 2024-08-08T20:48:47.5228325Z inflating: build/lib/torch/distributed/optim/zero_redundancy_optimizer.pyi 2024-08-08T20:48:47.5228898Z creating: build/lib/torch/distributed/pipelining/ 2024-08-08T20:48:47.5234584Z inflating: build/lib/torch/distributed/pipelining/_IR.py 2024-08-08T20:48:47.5235122Z inflating: build/lib/torch/distributed/pipelining/__init__.py 2024-08-08T20:48:47.5236742Z inflating: build/lib/torch/distributed/pipelining/_backward.py 2024-08-08T20:48:47.5237355Z inflating: build/lib/torch/distributed/pipelining/_debug.py 2024-08-08T20:48:47.5238116Z inflating: build/lib/torch/distributed/pipelining/_unflatten.py 2024-08-08T20:48:47.5239159Z inflating: build/lib/torch/distributed/pipelining/_utils.py 2024-08-08T20:48:47.5241510Z inflating: build/lib/torch/distributed/pipelining/microbatch.py 2024-08-08T20:48:47.5248075Z inflating: build/lib/torch/distributed/pipelining/schedules.py 2024-08-08T20:48:47.5253532Z inflating: build/lib/torch/distributed/pipelining/stage.py 2024-08-08T20:48:47.5253786Z creating: build/lib/torch/distributed/rpc/ 2024-08-08T20:48:47.5255645Z inflating: build/lib/torch/distributed/rpc/__init__.py 2024-08-08T20:48:47.5256165Z inflating: build/lib/torch/distributed/rpc/_utils.py 2024-08-08T20:48:47.5260823Z inflating: build/lib/torch/distributed/rpc/api.py 2024-08-08T20:48:47.5263087Z inflating: build/lib/torch/distributed/rpc/backend_registry.py 2024-08-08T20:48:47.5263871Z inflating: build/lib/torch/distributed/rpc/constants.py 2024-08-08T20:48:47.5265588Z inflating: build/lib/torch/distributed/rpc/functions.py 2024-08-08T20:48:47.5267325Z inflating: build/lib/torch/distributed/rpc/internal.py 2024-08-08T20:48:47.5268639Z inflating: build/lib/torch/distributed/rpc/options.py 2024-08-08T20:48:47.5269646Z inflating: build/lib/torch/distributed/rpc/rref_proxy.py 2024-08-08T20:48:47.5272156Z inflating: build/lib/torch/distributed/rpc/server_process_global_profiler.py 2024-08-08T20:48:47.5272733Z creating: build/lib/torch/distributed/rpc/_testing/ 2024-08-08T20:48:47.5273193Z inflating: build/lib/torch/distributed/rpc/_testing/__init__.py 2024-08-08T20:48:47.5273941Z inflating: build/lib/torch/distributed/rpc/_testing/faulty_agent_backend_registry.py 2024-08-08T20:48:47.5274179Z creating: build/lib/torch/distributed/tensor/ 2024-08-08T20:48:47.5274970Z extracting: build/lib/torch/distributed/tensor/__init__.py 2024-08-08T20:48:47.5275469Z creating: build/lib/torch/distributed/tensor/parallel/ 2024-08-08T20:48:47.5276356Z inflating: build/lib/torch/distributed/tensor/parallel/__init__.py 2024-08-08T20:48:47.5277479Z inflating: build/lib/torch/distributed/tensor/parallel/_data_parallel_utils.py 2024-08-08T20:48:47.5278332Z inflating: build/lib/torch/distributed/tensor/parallel/_utils.py 2024-08-08T20:48:47.5279677Z inflating: build/lib/torch/distributed/tensor/parallel/api.py 2024-08-08T20:48:47.5280883Z inflating: build/lib/torch/distributed/tensor/parallel/ddp.py 2024-08-08T20:48:47.5283069Z inflating: build/lib/torch/distributed/tensor/parallel/fsdp.py 2024-08-08T20:48:47.5284182Z inflating: build/lib/torch/distributed/tensor/parallel/input_reshard.py 2024-08-08T20:48:47.5286693Z inflating: build/lib/torch/distributed/tensor/parallel/loss.py 2024-08-08T20:48:47.5289755Z inflating: build/lib/torch/distributed/tensor/parallel/style.py 2024-08-08T20:48:47.5290017Z creating: build/lib/torch/distributions/ 2024-08-08T20:48:47.5291210Z inflating: build/lib/torch/distributions/__init__.py 2024-08-08T20:48:47.5292483Z inflating: build/lib/torch/distributions/bernoulli.py 2024-08-08T20:48:47.5293565Z inflating: build/lib/torch/distributions/beta.py 2024-08-08T20:48:47.5294930Z inflating: build/lib/torch/distributions/binomial.py 2024-08-08T20:48:47.5296292Z inflating: build/lib/torch/distributions/categorical.py 2024-08-08T20:48:47.5297309Z inflating: build/lib/torch/distributions/cauchy.py 2024-08-08T20:48:47.5297913Z inflating: build/lib/torch/distributions/chi2.py 2024-08-08T20:48:47.5299898Z inflating: build/lib/torch/distributions/constraint_registry.py 2024-08-08T20:48:47.5302376Z inflating: build/lib/torch/distributions/constraints.py 2024-08-08T20:48:47.5304278Z inflating: build/lib/torch/distributions/continuous_bernoulli.py 2024-08-08T20:48:47.5305393Z inflating: build/lib/torch/distributions/dirichlet.py 2024-08-08T20:48:47.5307421Z inflating: build/lib/torch/distributions/distribution.py 2024-08-08T20:48:47.5308088Z inflating: build/lib/torch/distributions/exp_family.py 2024-08-08T20:48:47.5309188Z inflating: build/lib/torch/distributions/exponential.py 2024-08-08T20:48:47.5310668Z inflating: build/lib/torch/distributions/fishersnedecor.py 2024-08-08T20:48:47.5311359Z inflating: build/lib/torch/distributions/gamma.py 2024-08-08T20:48:47.5312979Z inflating: build/lib/torch/distributions/geometric.py 2024-08-08T20:48:47.5314004Z inflating: build/lib/torch/distributions/gumbel.py 2024-08-08T20:48:47.5315197Z inflating: build/lib/torch/distributions/half_cauchy.py 2024-08-08T20:48:47.5316197Z inflating: build/lib/torch/distributions/half_normal.py 2024-08-08T20:48:47.5317469Z inflating: build/lib/torch/distributions/independent.py 2024-08-08T20:48:47.5318491Z inflating: build/lib/torch/distributions/inverse_gamma.py 2024-08-08T20:48:47.5321979Z inflating: build/lib/torch/distributions/kl.py 2024-08-08T20:48:47.5323120Z inflating: build/lib/torch/distributions/kumaraswamy.py 2024-08-08T20:48:47.5324381Z inflating: build/lib/torch/distributions/laplace.py 2024-08-08T20:48:47.5325724Z inflating: build/lib/torch/distributions/lkj_cholesky.py 2024-08-08T20:48:47.5326699Z inflating: build/lib/torch/distributions/log_normal.py 2024-08-08T20:48:47.5327768Z inflating: build/lib/torch/distributions/logistic_normal.py 2024-08-08T20:48:47.5329705Z inflating: build/lib/torch/distributions/lowrank_multivariate_normal.py 2024-08-08T20:48:47.5331073Z inflating: build/lib/torch/distributions/mixture_same_family.py 2024-08-08T20:48:47.5332370Z inflating: build/lib/torch/distributions/multinomial.py 2024-08-08T20:48:47.5334329Z inflating: build/lib/torch/distributions/multivariate_normal.py 2024-08-08T20:48:47.5335466Z inflating: build/lib/torch/distributions/negative_binomial.py 2024-08-08T20:48:47.5336547Z inflating: build/lib/torch/distributions/normal.py 2024-08-08T20:48:47.5337935Z inflating: build/lib/torch/distributions/one_hot_categorical.py 2024-08-08T20:48:47.5338864Z inflating: build/lib/torch/distributions/pareto.py 2024-08-08T20:48:47.5339882Z inflating: build/lib/torch/distributions/poisson.py 2024-08-08T20:48:47.5341278Z inflating: build/lib/torch/distributions/relaxed_bernoulli.py 2024-08-08T20:48:47.5342657Z inflating: build/lib/torch/distributions/relaxed_categorical.py 2024-08-08T20:48:47.5343637Z inflating: build/lib/torch/distributions/studentT.py 2024-08-08T20:48:47.5345731Z inflating: build/lib/torch/distributions/transformed_distribution.py 2024-08-08T20:48:47.5349739Z inflating: build/lib/torch/distributions/transforms.py 2024-08-08T20:48:47.5350736Z inflating: build/lib/torch/distributions/uniform.py 2024-08-08T20:48:47.5352300Z inflating: build/lib/torch/distributions/utils.py 2024-08-08T20:48:47.5353745Z inflating: build/lib/torch/distributions/von_mises.py 2024-08-08T20:48:47.5354824Z inflating: build/lib/torch/distributions/weibull.py 2024-08-08T20:48:47.5357286Z inflating: build/lib/torch/distributions/wishart.py 2024-08-08T20:48:47.5357500Z creating: build/lib/torch/export/ 2024-08-08T20:48:47.5360173Z inflating: build/lib/torch/export/__init__.py 2024-08-08T20:48:47.5361293Z inflating: build/lib/torch/export/_remove_auto_functionalized_pass.py 2024-08-08T20:48:47.5362676Z inflating: build/lib/torch/export/_remove_effect_tokens_pass.py 2024-08-08T20:48:47.5363539Z inflating: build/lib/torch/export/_safeguard.py 2024-08-08T20:48:47.5371332Z inflating: build/lib/torch/export/_trace.py 2024-08-08T20:48:47.5372053Z inflating: build/lib/torch/export/_tree_utils.py 2024-08-08T20:48:47.5374066Z inflating: build/lib/torch/export/_unlift.py 2024-08-08T20:48:47.5374806Z inflating: build/lib/torch/export/custom_obj.py 2024-08-08T20:48:47.5379629Z inflating: build/lib/torch/export/dynamic_shapes.py 2024-08-08T20:48:47.5384715Z inflating: build/lib/torch/export/exported_program.py 2024-08-08T20:48:47.5387137Z inflating: build/lib/torch/export/graph_signature.py 2024-08-08T20:48:47.5392601Z inflating: build/lib/torch/export/unflatten.py 2024-08-08T20:48:47.5393310Z creating: build/lib/torch/export/experimental/ 2024-08-08T20:48:47.5394074Z inflating: build/lib/torch/export/experimental/__init__.py 2024-08-08T20:48:47.5394764Z creating: build/lib/torch/fft/ 2024-08-08T20:48:47.5398728Z inflating: build/lib/torch/fft/__init__.py 2024-08-08T20:48:47.5399325Z creating: build/lib/torch/func/ 2024-08-08T20:48:47.5399886Z inflating: build/lib/torch/func/__init__.py 2024-08-08T20:48:47.5401039Z creating: build/lib/torch/futures/ 2024-08-08T20:48:47.5402883Z inflating: build/lib/torch/futures/__init__.py 2024-08-08T20:48:47.5403511Z creating: build/lib/torch/fx/ 2024-08-08T20:48:47.5404262Z inflating: build/lib/torch/fx/__init__.py 2024-08-08T20:48:47.5405332Z inflating: build/lib/torch/fx/_compatibility.py 2024-08-08T20:48:47.5407051Z inflating: build/lib/torch/fx/_lazy_graph_module.py 2024-08-08T20:48:47.5408223Z inflating: build/lib/torch/fx/_pytree.py 2024-08-08T20:48:47.5413500Z inflating: build/lib/torch/fx/_symbolic_trace.py 2024-08-08T20:48:47.5414659Z inflating: build/lib/torch/fx/_utils.py 2024-08-08T20:48:47.5415685Z inflating: build/lib/torch/fx/annotate.py 2024-08-08T20:48:47.5416392Z inflating: build/lib/torch/fx/config.py 2024-08-08T20:48:47.5423444Z inflating: build/lib/torch/fx/graph.py 2024-08-08T20:48:47.5428061Z inflating: build/lib/torch/fx/graph_module.py 2024-08-08T20:48:47.5429311Z inflating: build/lib/torch/fx/immutable_collections.py 2024-08-08T20:48:47.5431898Z inflating: build/lib/torch/fx/interpreter.py 2024-08-08T20:48:47.5436510Z inflating: build/lib/torch/fx/node.py 2024-08-08T20:48:47.5439335Z inflating: build/lib/torch/fx/operator_schemas.py 2024-08-08T20:48:47.5442212Z inflating: build/lib/torch/fx/proxy.py 2024-08-08T20:48:47.5444402Z inflating: build/lib/torch/fx/subgraph_rewriter.py 2024-08-08T20:48:47.5445472Z inflating: build/lib/torch/fx/tensor_type.py 2024-08-08T20:48:47.5446456Z inflating: build/lib/torch/fx/traceback.py 2024-08-08T20:48:47.5447416Z creating: build/lib/torch/fx/experimental/ 2024-08-08T20:48:47.5448498Z extracting: build/lib/torch/fx/experimental/__init__.py 2024-08-08T20:48:47.5449656Z inflating: build/lib/torch/fx/experimental/_backward_state.py 2024-08-08T20:48:47.5450836Z inflating: build/lib/torch/fx/experimental/_config.py 2024-08-08T20:48:47.5451683Z inflating: build/lib/torch/fx/experimental/_sym_dispatch_mode.py 2024-08-08T20:48:47.5455586Z inflating: build/lib/torch/fx/experimental/accelerator_partitioner.py 2024-08-08T20:48:47.5457393Z inflating: build/lib/torch/fx/experimental/const_fold.py 2024-08-08T20:48:47.5458833Z inflating: build/lib/torch/fx/experimental/debug.py 2024-08-08T20:48:47.5462744Z inflating: build/lib/torch/fx/experimental/graph_gradual_typechecker.py 2024-08-08T20:48:47.5464702Z inflating: build/lib/torch/fx/experimental/merge_matmul.py 2024-08-08T20:48:47.5466463Z inflating: build/lib/torch/fx/experimental/meta_tracer.py 2024-08-08T20:48:47.5467843Z inflating: build/lib/torch/fx/experimental/normalize.py 2024-08-08T20:48:47.5470414Z inflating: build/lib/torch/fx/experimental/optimization.py 2024-08-08T20:48:47.5472288Z inflating: build/lib/torch/fx/experimental/partitioner_utils.py 2024-08-08T20:48:47.5479857Z inflating: build/lib/torch/fx/experimental/proxy_tensor.py 2024-08-08T20:48:47.5482603Z inflating: build/lib/torch/fx/experimental/recording.py 2024-08-08T20:48:47.5483757Z inflating: build/lib/torch/fx/experimental/refinement_types.py 2024-08-08T20:48:47.5485002Z inflating: build/lib/torch/fx/experimental/rewriter.py 2024-08-08T20:48:47.5486330Z inflating: build/lib/torch/fx/experimental/schema_type_annotation.py 2024-08-08T20:48:47.5491284Z inflating: build/lib/torch/fx/experimental/sym_node.py 2024-08-08T20:48:47.5514513Z inflating: build/lib/torch/fx/experimental/symbolic_shapes.py 2024-08-08T20:48:47.5515805Z inflating: build/lib/torch/fx/experimental/unify_refinements.py 2024-08-08T20:48:47.5519793Z inflating: build/lib/torch/fx/experimental/validator.py 2024-08-08T20:48:47.5520874Z creating: build/lib/torch/fx/experimental/migrate_gradual_types/ 2024-08-08T20:48:47.5522020Z extracting: build/lib/torch/fx/experimental/migrate_gradual_types/__init__.py 2024-08-08T20:48:47.5523393Z inflating: build/lib/torch/fx/experimental/migrate_gradual_types/constraint.py 2024-08-08T20:48:47.5527435Z inflating: build/lib/torch/fx/experimental/migrate_gradual_types/constraint_generator.py 2024-08-08T20:48:47.5531751Z inflating: build/lib/torch/fx/experimental/migrate_gradual_types/constraint_transformation.py 2024-08-08T20:48:47.5533100Z inflating: build/lib/torch/fx/experimental/migrate_gradual_types/operation.py 2024-08-08T20:48:47.5534822Z inflating: build/lib/torch/fx/experimental/migrate_gradual_types/transform_to_z3.py 2024-08-08T20:48:47.5536706Z inflating: build/lib/torch/fx/experimental/migrate_gradual_types/util.py 2024-08-08T20:48:47.5538370Z inflating: build/lib/torch/fx/experimental/migrate_gradual_types/z3_types.py 2024-08-08T20:48:47.5539632Z creating: build/lib/torch/fx/experimental/unification/ 2024-08-08T20:48:47.5540956Z inflating: build/lib/torch/fx/experimental/unification/__init__.py 2024-08-08T20:48:47.5542069Z inflating: build/lib/torch/fx/experimental/unification/core.py 2024-08-08T20:48:47.5543338Z inflating: build/lib/torch/fx/experimental/unification/dispatch.py 2024-08-08T20:48:47.5544816Z inflating: build/lib/torch/fx/experimental/unification/match.py 2024-08-08T20:48:47.5545936Z inflating: build/lib/torch/fx/experimental/unification/more.py 2024-08-08T20:48:47.5547501Z inflating: build/lib/torch/fx/experimental/unification/unification_tools.py 2024-08-08T20:48:47.5548963Z inflating: build/lib/torch/fx/experimental/unification/utils.py 2024-08-08T20:48:47.5550181Z inflating: build/lib/torch/fx/experimental/unification/variable.py 2024-08-08T20:48:47.5551427Z creating: build/lib/torch/fx/experimental/unification/multipledispatch/ 2024-08-08T20:48:47.5553060Z inflating: build/lib/torch/fx/experimental/unification/multipledispatch/__init__.py 2024-08-08T20:48:47.5554952Z inflating: build/lib/torch/fx/experimental/unification/multipledispatch/conflict.py 2024-08-08T20:48:47.5556751Z inflating: build/lib/torch/fx/experimental/unification/multipledispatch/core.py 2024-08-08T20:48:47.5558744Z inflating: build/lib/torch/fx/experimental/unification/multipledispatch/dispatcher.py 2024-08-08T20:48:47.5560450Z inflating: build/lib/torch/fx/experimental/unification/multipledispatch/utils.py 2024-08-08T20:48:47.5562571Z inflating: build/lib/torch/fx/experimental/unification/multipledispatch/variadic.py 2024-08-08T20:48:47.5563609Z creating: build/lib/torch/fx/passes/ 2024-08-08T20:48:47.5564242Z inflating: build/lib/torch/fx/passes/__init__.py 2024-08-08T20:48:47.5565037Z inflating: build/lib/torch/fx/passes/annotate_getitem_nodes.py 2024-08-08T20:48:47.5565905Z inflating: build/lib/torch/fx/passes/fake_tensor_prop.py 2024-08-08T20:48:47.5567007Z inflating: build/lib/torch/fx/passes/graph_drawer.py 2024-08-08T20:48:47.5568132Z inflating: build/lib/torch/fx/passes/graph_manipulation.py 2024-08-08T20:48:47.5569396Z inflating: build/lib/torch/fx/passes/graph_transform_observer.py 2024-08-08T20:48:47.5570289Z inflating: build/lib/torch/fx/passes/net_min_base.py 2024-08-08T20:48:47.5571344Z inflating: build/lib/torch/fx/passes/operator_support.py 2024-08-08T20:48:47.5572105Z inflating: build/lib/torch/fx/passes/param_fetch.py 2024-08-08T20:48:47.5572867Z inflating: build/lib/torch/fx/passes/pass_manager.py 2024-08-08T20:48:47.5575127Z inflating: build/lib/torch/fx/passes/reinplace.py 2024-08-08T20:48:47.5578296Z inflating: build/lib/torch/fx/passes/runtime_assert.py 2024-08-08T20:48:47.5579683Z inflating: build/lib/torch/fx/passes/shape_prop.py 2024-08-08T20:48:47.5582834Z inflating: build/lib/torch/fx/passes/split_module.py 2024-08-08T20:48:47.5584650Z inflating: build/lib/torch/fx/passes/split_utils.py 2024-08-08T20:48:47.5588403Z inflating: build/lib/torch/fx/passes/splitter_base.py 2024-08-08T20:48:47.5590167Z inflating: build/lib/torch/fx/passes/tools_common.py 2024-08-08T20:48:47.5591174Z creating: build/lib/torch/fx/passes/backends/ 2024-08-08T20:48:47.5592273Z extracting: build/lib/torch/fx/passes/backends/__init__.py 2024-08-08T20:48:47.5593503Z inflating: build/lib/torch/fx/passes/backends/cudagraphs.py 2024-08-08T20:48:47.5594490Z creating: build/lib/torch/fx/passes/dialect/ 2024-08-08T20:48:47.5595598Z extracting: build/lib/torch/fx/passes/dialect/__init__.py 2024-08-08T20:48:47.5596842Z creating: build/lib/torch/fx/passes/dialect/common/ 2024-08-08T20:48:47.5597695Z extracting: build/lib/torch/fx/passes/dialect/common/__init__.py 2024-08-08T20:48:47.5598943Z inflating: build/lib/torch/fx/passes/dialect/common/cse_pass.py 2024-08-08T20:48:47.5600098Z creating: build/lib/torch/fx/passes/infra/ 2024-08-08T20:48:47.5600788Z extracting: build/lib/torch/fx/passes/infra/__init__.py 2024-08-08T20:48:47.5601998Z inflating: build/lib/torch/fx/passes/infra/partitioner.py 2024-08-08T20:48:47.5603096Z inflating: build/lib/torch/fx/passes/infra/pass_base.py 2024-08-08T20:48:47.5604337Z inflating: build/lib/torch/fx/passes/infra/pass_manager.py 2024-08-08T20:48:47.5605475Z creating: build/lib/torch/fx/passes/tests/ 2024-08-08T20:48:47.5606507Z extracting: build/lib/torch/fx/passes/tests/__init__.py 2024-08-08T20:48:47.5607348Z inflating: build/lib/torch/fx/passes/tests/test_pass_manager.py 2024-08-08T20:48:47.5608497Z creating: build/lib/torch/fx/passes/utils/ 2024-08-08T20:48:47.5609175Z inflating: build/lib/torch/fx/passes/utils/__init__.py 2024-08-08T20:48:47.5610019Z inflating: build/lib/torch/fx/passes/utils/common.py 2024-08-08T20:48:47.5611076Z inflating: build/lib/torch/fx/passes/utils/fuser_utils.py 2024-08-08T20:48:47.5612258Z inflating: build/lib/torch/fx/passes/utils/matcher_utils.py 2024-08-08T20:48:47.5613860Z inflating: build/lib/torch/fx/passes/utils/matcher_with_name_node_map_utils.py 2024-08-08T20:48:47.5615112Z inflating: build/lib/torch/fx/passes/utils/source_matcher_utils.py 2024-08-08T20:48:47.5616258Z inflating: build/lib/torch/fx/__init__.pyi 2024-08-08T20:48:47.5617117Z creating: build/lib/torch/jit/ 2024-08-08T20:48:47.5617982Z inflating: build/lib/torch/jit/__init__.py 2024-08-08T20:48:47.5618634Z inflating: build/lib/torch/jit/_async.py 2024-08-08T20:48:47.5619583Z inflating: build/lib/torch/jit/_await.py 2024-08-08T20:48:47.5620352Z inflating: build/lib/torch/jit/_builtins.py 2024-08-08T20:48:47.5621348Z inflating: build/lib/torch/jit/_check.py 2024-08-08T20:48:47.5622655Z inflating: build/lib/torch/jit/_dataclass_impls.py 2024-08-08T20:48:47.5623712Z inflating: build/lib/torch/jit/_decomposition_utils.py 2024-08-08T20:48:47.5624941Z inflating: build/lib/torch/jit/_decompositions.py 2024-08-08T20:48:47.5626588Z inflating: build/lib/torch/jit/_freeze.py 2024-08-08T20:48:47.5628115Z inflating: build/lib/torch/jit/_fuser.py 2024-08-08T20:48:47.5629228Z inflating: build/lib/torch/jit/_ir_utils.py 2024-08-08T20:48:47.5630079Z inflating: build/lib/torch/jit/_logging.py 2024-08-08T20:48:47.5631317Z inflating: build/lib/torch/jit/_monkeytype_config.py 2024-08-08T20:48:47.5632380Z inflating: build/lib/torch/jit/_pickle.py 2024-08-08T20:48:47.5636752Z inflating: build/lib/torch/jit/_recursive.py 2024-08-08T20:48:47.5643118Z inflating: build/lib/torch/jit/_script.py 2024-08-08T20:48:47.5644749Z inflating: build/lib/torch/jit/_serialization.py 2024-08-08T20:48:47.5649388Z inflating: build/lib/torch/jit/_shape_functions.py 2024-08-08T20:48:47.5650531Z inflating: build/lib/torch/jit/_state.py 2024-08-08T20:48:47.5656172Z inflating: build/lib/torch/jit/_trace.py 2024-08-08T20:48:47.5658929Z inflating: build/lib/torch/jit/annotations.py 2024-08-08T20:48:47.5663764Z inflating: build/lib/torch/jit/frontend.py 2024-08-08T20:48:47.5664823Z inflating: build/lib/torch/jit/generate_bytecode.py 2024-08-08T20:48:47.5665903Z inflating: build/lib/torch/jit/quantized.py 2024-08-08T20:48:47.5667472Z inflating: build/lib/torch/jit/supported_ops.py 2024-08-08T20:48:47.5668584Z inflating: build/lib/torch/jit/unsupported_tensor_ops.py 2024-08-08T20:48:47.5669587Z creating: build/lib/torch/jit/_passes/ 2024-08-08T20:48:47.5670673Z extracting: build/lib/torch/jit/_passes/__init__.py 2024-08-08T20:48:47.5671631Z inflating: build/lib/torch/jit/_passes/_property_propagation.py 2024-08-08T20:48:47.5672756Z creating: build/lib/torch/jit/mobile/ 2024-08-08T20:48:47.5673394Z inflating: build/lib/torch/jit/mobile/__init__.py 2024-08-08T20:48:47.5674578Z inflating: build/lib/torch/jit/_script.pyi 2024-08-08T20:48:47.5675153Z creating: build/lib/torch/linalg/ 2024-08-08T20:48:47.5684956Z inflating: build/lib/torch/linalg/__init__.py 2024-08-08T20:48:47.5685978Z creating: build/lib/torch/masked/ 2024-08-08T20:48:47.5686570Z inflating: build/lib/torch/masked/__init__.py 2024-08-08T20:48:47.5689547Z inflating: build/lib/torch/masked/_docs.py 2024-08-08T20:48:47.5695501Z inflating: build/lib/torch/masked/_ops.py 2024-08-08T20:48:47.5696553Z creating: build/lib/torch/masked/maskedtensor/ 2024-08-08T20:48:47.5697276Z inflating: build/lib/torch/masked/maskedtensor/__init__.py 2024-08-08T20:48:47.5698727Z inflating: build/lib/torch/masked/maskedtensor/_ops_refs.py 2024-08-08T20:48:47.5700009Z inflating: build/lib/torch/masked/maskedtensor/binary.py 2024-08-08T20:48:47.5702353Z inflating: build/lib/torch/masked/maskedtensor/core.py 2024-08-08T20:48:47.5703646Z inflating: build/lib/torch/masked/maskedtensor/creation.py 2024-08-08T20:48:47.5704822Z inflating: build/lib/torch/masked/maskedtensor/passthrough.py 2024-08-08T20:48:47.5706076Z inflating: build/lib/torch/masked/maskedtensor/reductions.py 2024-08-08T20:48:47.5707430Z inflating: build/lib/torch/masked/maskedtensor/unary.py 2024-08-08T20:48:47.5708277Z creating: build/lib/torch/monitor/ 2024-08-08T20:48:47.5709241Z inflating: build/lib/torch/monitor/__init__.py 2024-08-08T20:48:47.5710146Z creating: build/lib/torch/mps/ 2024-08-08T20:48:47.5711027Z inflating: build/lib/torch/mps/__init__.py 2024-08-08T20:48:47.5711789Z inflating: build/lib/torch/mps/event.py 2024-08-08T20:48:47.5712392Z inflating: build/lib/torch/mps/profiler.py 2024-08-08T20:48:47.5713290Z creating: build/lib/torch/mtia/ 2024-08-08T20:48:47.5714825Z inflating: build/lib/torch/mtia/__init__.py 2024-08-08T20:48:47.5715803Z inflating: build/lib/torch/mtia/_utils.py 2024-08-08T20:48:47.5716737Z creating: build/lib/torch/multiprocessing/ 2024-08-08T20:48:47.5717913Z inflating: build/lib/torch/multiprocessing/__init__.py 2024-08-08T20:48:47.5718920Z inflating: build/lib/torch/multiprocessing/_atfork.py 2024-08-08T20:48:47.5719646Z inflating: build/lib/torch/multiprocessing/pool.py 2024-08-08T20:48:47.5720383Z inflating: build/lib/torch/multiprocessing/queue.py 2024-08-08T20:48:47.5721535Z inflating: build/lib/torch/multiprocessing/reductions.py 2024-08-08T20:48:47.5723162Z inflating: build/lib/torch/multiprocessing/spawn.py 2024-08-08T20:48:47.5723985Z creating: build/lib/torch/nested/ 2024-08-08T20:48:47.5726258Z inflating: build/lib/torch/nested/__init__.py 2024-08-08T20:48:47.5727180Z creating: build/lib/torch/nested/_internal/ 2024-08-08T20:48:47.5727866Z extracting: build/lib/torch/nested/_internal/__init__.py 2024-08-08T20:48:47.5729533Z inflating: build/lib/torch/nested/_internal/nested_tensor.py 2024-08-08T20:48:47.5734557Z inflating: build/lib/torch/nested/_internal/ops.py 2024-08-08T20:48:47.5738024Z inflating: build/lib/torch/nested/_internal/sdpa.py 2024-08-08T20:48:47.5739114Z creating: build/lib/torch/nn/ 2024-08-08T20:48:47.5740019Z inflating: build/lib/torch/nn/__init__.py 2024-08-08T20:48:47.5741026Z inflating: build/lib/torch/nn/_reduction.py 2024-08-08T20:48:47.5742014Z inflating: build/lib/torch/nn/common_types.py 2024-08-08T20:48:47.5742638Z inflating: build/lib/torch/nn/cpp.py 2024-08-08T20:48:47.5760502Z inflating: build/lib/torch/nn/functional.py 2024-08-08T20:48:47.5761643Z inflating: build/lib/torch/nn/grad.py 2024-08-08T20:48:47.5764583Z inflating: build/lib/torch/nn/init.py 2024-08-08T20:48:47.5766271Z inflating: build/lib/torch/nn/parameter.py 2024-08-08T20:48:47.5767042Z creating: build/lib/torch/nn/attention/ 2024-08-08T20:48:47.5768076Z inflating: build/lib/torch/nn/attention/__init__.py 2024-08-08T20:48:47.5769169Z inflating: build/lib/torch/nn/attention/_utils.py 2024-08-08T20:48:47.5770809Z inflating: build/lib/torch/nn/attention/bias.py 2024-08-08T20:48:47.5775096Z inflating: build/lib/torch/nn/attention/flex_attention.py 2024-08-08T20:48:47.5776124Z creating: build/lib/torch/nn/backends/ 2024-08-08T20:48:47.5777175Z extracting: build/lib/torch/nn/backends/__init__.py 2024-08-08T20:48:47.5778325Z inflating: build/lib/torch/nn/backends/thnn.py 2024-08-08T20:48:47.5779442Z creating: build/lib/torch/nn/intrinsic/ 2024-08-08T20:48:47.5780441Z inflating: build/lib/torch/nn/intrinsic/__init__.py 2024-08-08T20:48:47.5781618Z creating: build/lib/torch/nn/intrinsic/modules/ 2024-08-08T20:48:47.5782827Z inflating: build/lib/torch/nn/intrinsic/modules/__init__.py 2024-08-08T20:48:47.5784058Z inflating: build/lib/torch/nn/intrinsic/modules/fused.py 2024-08-08T20:48:47.5785223Z creating: build/lib/torch/nn/intrinsic/qat/ 2024-08-08T20:48:47.5786348Z extracting: build/lib/torch/nn/intrinsic/qat/__init__.py 2024-08-08T20:48:47.5787570Z creating: build/lib/torch/nn/intrinsic/qat/modules/ 2024-08-08T20:48:47.5788849Z inflating: build/lib/torch/nn/intrinsic/qat/modules/__init__.py 2024-08-08T20:48:47.5790337Z inflating: build/lib/torch/nn/intrinsic/qat/modules/conv_fused.py 2024-08-08T20:48:47.5791932Z inflating: build/lib/torch/nn/intrinsic/qat/modules/linear_fused.py 2024-08-08T20:48:47.5793536Z inflating: build/lib/torch/nn/intrinsic/qat/modules/linear_relu.py 2024-08-08T20:48:47.5794639Z creating: build/lib/torch/nn/intrinsic/quantized/ 2024-08-08T20:48:47.5795795Z inflating: build/lib/torch/nn/intrinsic/quantized/__init__.py 2024-08-08T20:48:47.5796641Z creating: build/lib/torch/nn/intrinsic/quantized/dynamic/ 2024-08-08T20:48:47.5797923Z extracting: build/lib/torch/nn/intrinsic/quantized/dynamic/__init__.py 2024-08-08T20:48:47.5798882Z creating: build/lib/torch/nn/intrinsic/quantized/dynamic/modules/ 2024-08-08T20:48:47.5800151Z inflating: build/lib/torch/nn/intrinsic/quantized/dynamic/modules/__init__.py 2024-08-08T20:48:47.5801351Z inflating: build/lib/torch/nn/intrinsic/quantized/dynamic/modules/linear_relu.py 2024-08-08T20:48:47.5802672Z creating: build/lib/torch/nn/intrinsic/quantized/modules/ 2024-08-08T20:48:47.5803600Z inflating: build/lib/torch/nn/intrinsic/quantized/modules/__init__.py 2024-08-08T20:48:47.5804986Z inflating: build/lib/torch/nn/intrinsic/quantized/modules/bn_relu.py 2024-08-08T20:48:47.5806003Z inflating: build/lib/torch/nn/intrinsic/quantized/modules/conv_relu.py 2024-08-08T20:48:47.5807068Z inflating: build/lib/torch/nn/intrinsic/quantized/modules/linear_relu.py 2024-08-08T20:48:47.5808265Z creating: build/lib/torch/nn/modules/ 2024-08-08T20:48:47.5809150Z inflating: build/lib/torch/nn/modules/__init__.py 2024-08-08T20:48:47.5809929Z inflating: build/lib/torch/nn/modules/_functions.py 2024-08-08T20:48:47.5810694Z inflating: build/lib/torch/nn/modules/activation.py 2024-08-08T20:48:47.5811406Z inflating: build/lib/torch/nn/modules/adaptive.py 2024-08-08T20:48:47.5812282Z inflating: build/lib/torch/nn/modules/batchnorm.py 2024-08-08T20:48:47.5813375Z inflating: build/lib/torch/nn/modules/channelshuffle.py 2024-08-08T20:48:47.5814134Z inflating: build/lib/torch/nn/modules/container.py 2024-08-08T20:48:47.5818809Z inflating: build/lib/torch/nn/modules/conv.py 2024-08-08T20:48:47.5819824Z inflating: build/lib/torch/nn/modules/distance.py 2024-08-08T20:48:47.5821722Z inflating: build/lib/torch/nn/modules/dropout.py 2024-08-08T20:48:47.5823103Z inflating: build/lib/torch/nn/modules/flatten.py 2024-08-08T20:48:47.5825192Z inflating: build/lib/torch/nn/modules/fold.py 2024-08-08T20:48:47.5827327Z inflating: build/lib/torch/nn/modules/instancenorm.py 2024-08-08T20:48:47.5829085Z inflating: build/lib/torch/nn/modules/lazy.py 2024-08-08T20:48:47.5830779Z inflating: build/lib/torch/nn/modules/linear.py 2024-08-08T20:48:47.5838591Z inflating: build/lib/torch/nn/modules/loss.py 2024-08-08T20:48:47.5848937Z inflating: build/lib/torch/nn/modules/module.py 2024-08-08T20:48:47.5851095Z inflating: build/lib/torch/nn/modules/normalization.py 2024-08-08T20:48:47.5853571Z inflating: build/lib/torch/nn/modules/padding.py 2024-08-08T20:48:47.5854671Z inflating: build/lib/torch/nn/modules/pixelshuffle.py 2024-08-08T20:48:47.5859575Z inflating: build/lib/torch/nn/modules/pooling.py 2024-08-08T20:48:47.5865945Z inflating: build/lib/torch/nn/modules/rnn.py 2024-08-08T20:48:47.5869292Z inflating: build/lib/torch/nn/modules/sparse.py 2024-08-08T20:48:47.5873874Z inflating: build/lib/torch/nn/modules/transformer.py 2024-08-08T20:48:47.5875655Z inflating: build/lib/torch/nn/modules/upsampling.py 2024-08-08T20:48:47.5876713Z inflating: build/lib/torch/nn/modules/utils.py 2024-08-08T20:48:47.5877764Z creating: build/lib/torch/nn/parallel/ 2024-08-08T20:48:47.5878587Z inflating: build/lib/torch/nn/parallel/__init__.py 2024-08-08T20:48:47.5879602Z inflating: build/lib/torch/nn/parallel/_functions.py 2024-08-08T20:48:47.5881091Z inflating: build/lib/torch/nn/parallel/comm.py 2024-08-08T20:48:47.5883221Z inflating: build/lib/torch/nn/parallel/data_parallel.py 2024-08-08T20:48:47.5893538Z inflating: build/lib/torch/nn/parallel/distributed.py 2024-08-08T20:48:47.5894656Z inflating: build/lib/torch/nn/parallel/parallel_apply.py 2024-08-08T20:48:47.5896048Z inflating: build/lib/torch/nn/parallel/replicate.py 2024-08-08T20:48:47.5897492Z inflating: build/lib/torch/nn/parallel/scatter_gather.py 2024-08-08T20:48:47.5898693Z creating: build/lib/torch/nn/qat/ 2024-08-08T20:48:47.5899553Z inflating: build/lib/torch/nn/qat/__init__.py 2024-08-08T20:48:47.5900608Z creating: build/lib/torch/nn/qat/dynamic/ 2024-08-08T20:48:47.5901758Z inflating: build/lib/torch/nn/qat/dynamic/__init__.py 2024-08-08T20:48:47.5903147Z creating: build/lib/torch/nn/qat/dynamic/modules/ 2024-08-08T20:48:47.5904420Z inflating: build/lib/torch/nn/qat/dynamic/modules/__init__.py 2024-08-08T20:48:47.5905848Z inflating: build/lib/torch/nn/qat/dynamic/modules/linear.py 2024-08-08T20:48:47.5907030Z creating: build/lib/torch/nn/qat/modules/ 2024-08-08T20:48:47.5908075Z inflating: build/lib/torch/nn/qat/modules/__init__.py 2024-08-08T20:48:47.5909223Z inflating: build/lib/torch/nn/qat/modules/conv.py 2024-08-08T20:48:47.5910480Z inflating: build/lib/torch/nn/qat/modules/embedding_ops.py 2024-08-08T20:48:47.5911802Z inflating: build/lib/torch/nn/qat/modules/linear.py 2024-08-08T20:48:47.5912899Z creating: build/lib/torch/nn/quantizable/ 2024-08-08T20:48:47.5913913Z extracting: build/lib/torch/nn/quantizable/__init__.py 2024-08-08T20:48:47.5915146Z creating: build/lib/torch/nn/quantizable/modules/ 2024-08-08T20:48:47.5916416Z inflating: build/lib/torch/nn/quantizable/modules/__init__.py 2024-08-08T20:48:47.5917822Z inflating: build/lib/torch/nn/quantizable/modules/activation.py 2024-08-08T20:48:47.5919113Z inflating: build/lib/torch/nn/quantizable/modules/rnn.py 2024-08-08T20:48:47.5920494Z creating: build/lib/torch/nn/quantized/ 2024-08-08T20:48:47.5921605Z inflating: build/lib/torch/nn/quantized/__init__.py 2024-08-08T20:48:47.5922711Z inflating: build/lib/torch/nn/quantized/functional.py 2024-08-08T20:48:47.5923875Z creating: build/lib/torch/nn/quantized/_reference/ 2024-08-08T20:48:47.5925114Z extracting: build/lib/torch/nn/quantized/_reference/__init__.py 2024-08-08T20:48:47.5926481Z creating: build/lib/torch/nn/quantized/_reference/modules/ 2024-08-08T20:48:47.5928004Z inflating: build/lib/torch/nn/quantized/_reference/modules/__init__.py 2024-08-08T20:48:47.5929533Z inflating: build/lib/torch/nn/quantized/_reference/modules/conv.py 2024-08-08T20:48:47.5930659Z inflating: build/lib/torch/nn/quantized/_reference/modules/linear.py 2024-08-08T20:48:47.5932177Z inflating: build/lib/torch/nn/quantized/_reference/modules/rnn.py 2024-08-08T20:48:47.5933557Z inflating: build/lib/torch/nn/quantized/_reference/modules/sparse.py 2024-08-08T20:48:47.5935121Z inflating: build/lib/torch/nn/quantized/_reference/modules/utils.py 2024-08-08T20:48:47.5936403Z creating: build/lib/torch/nn/quantized/dynamic/ 2024-08-08T20:48:47.5937642Z extracting: build/lib/torch/nn/quantized/dynamic/__init__.py 2024-08-08T20:48:47.5938532Z creating: build/lib/torch/nn/quantized/dynamic/modules/ 2024-08-08T20:48:47.5939732Z inflating: build/lib/torch/nn/quantized/dynamic/modules/__init__.py 2024-08-08T20:48:47.5941064Z inflating: build/lib/torch/nn/quantized/dynamic/modules/conv.py 2024-08-08T20:48:47.5942291Z inflating: build/lib/torch/nn/quantized/dynamic/modules/linear.py 2024-08-08T20:48:47.5943615Z inflating: build/lib/torch/nn/quantized/dynamic/modules/rnn.py 2024-08-08T20:48:47.5944592Z creating: build/lib/torch/nn/quantized/modules/ 2024-08-08T20:48:47.5945624Z inflating: build/lib/torch/nn/quantized/modules/__init__.py 2024-08-08T20:48:47.5946769Z inflating: build/lib/torch/nn/quantized/modules/activation.py 2024-08-08T20:48:47.5947997Z inflating: build/lib/torch/nn/quantized/modules/batchnorm.py 2024-08-08T20:48:47.5948832Z inflating: build/lib/torch/nn/quantized/modules/conv.py 2024-08-08T20:48:47.5949628Z inflating: build/lib/torch/nn/quantized/modules/dropout.py 2024-08-08T20:48:47.5950776Z inflating: build/lib/torch/nn/quantized/modules/embedding_ops.py 2024-08-08T20:48:47.5951844Z inflating: build/lib/torch/nn/quantized/modules/functional_modules.py 2024-08-08T20:48:47.5952752Z inflating: build/lib/torch/nn/quantized/modules/linear.py 2024-08-08T20:48:47.5953600Z inflating: build/lib/torch/nn/quantized/modules/normalization.py 2024-08-08T20:48:47.5954745Z inflating: build/lib/torch/nn/quantized/modules/rnn.py 2024-08-08T20:48:47.5955598Z inflating: build/lib/torch/nn/quantized/modules/utils.py 2024-08-08T20:48:47.5956386Z creating: build/lib/torch/nn/utils/ 2024-08-08T20:48:47.5956984Z inflating: build/lib/torch/nn/utils/__init__.py 2024-08-08T20:48:47.5957723Z inflating: build/lib/torch/nn/utils/_deprecation_utils.py 2024-08-08T20:48:47.5958961Z inflating: build/lib/torch/nn/utils/_named_member_accessor.py 2024-08-08T20:48:47.5960002Z inflating: build/lib/torch/nn/utils/_per_sample_grad.py 2024-08-08T20:48:47.5960980Z inflating: build/lib/torch/nn/utils/clip_grad.py 2024-08-08T20:48:47.5961731Z inflating: build/lib/torch/nn/utils/convert_parameters.py 2024-08-08T20:48:47.5962438Z inflating: build/lib/torch/nn/utils/fusion.py 2024-08-08T20:48:47.5963434Z inflating: build/lib/torch/nn/utils/init.py 2024-08-08T20:48:47.5964119Z inflating: build/lib/torch/nn/utils/memory_format.py 2024-08-08T20:48:47.5965009Z inflating: build/lib/torch/nn/utils/parametrizations.py 2024-08-08T20:48:47.5966006Z inflating: build/lib/torch/nn/utils/parametrize.py 2024-08-08T20:48:47.5966717Z inflating: build/lib/torch/nn/utils/prune.py 2024-08-08T20:48:47.5967695Z inflating: build/lib/torch/nn/utils/rnn.py 2024-08-08T20:48:47.5968671Z inflating: build/lib/torch/nn/utils/spectral_norm.py 2024-08-08T20:48:47.5969869Z inflating: build/lib/torch/nn/utils/stateless.py 2024-08-08T20:48:47.5971011Z inflating: build/lib/torch/nn/utils/weight_norm.py 2024-08-08T20:48:47.5971923Z creating: build/lib/torch/nn/utils/_expanded_weights/ 2024-08-08T20:48:47.5973034Z inflating: build/lib/torch/nn/utils/_expanded_weights/__init__.py 2024-08-08T20:48:47.5974506Z inflating: build/lib/torch/nn/utils/_expanded_weights/conv_expanded_weights.py 2024-08-08T20:48:47.5975867Z inflating: build/lib/torch/nn/utils/_expanded_weights/conv_utils.py 2024-08-08T20:48:47.5977186Z inflating: build/lib/torch/nn/utils/_expanded_weights/embedding_expanded_weights.py 2024-08-08T20:48:47.5978825Z inflating: build/lib/torch/nn/utils/_expanded_weights/expanded_weights_impl.py 2024-08-08T20:48:47.5980493Z inflating: build/lib/torch/nn/utils/_expanded_weights/expanded_weights_utils.py 2024-08-08T20:48:47.5982044Z inflating: build/lib/torch/nn/utils/_expanded_weights/group_norm_expanded_weights.py 2024-08-08T20:48:47.5983976Z inflating: build/lib/torch/nn/utils/_expanded_weights/instance_norm_expanded_weights.py 2024-08-08T20:48:47.5985716Z inflating: build/lib/torch/nn/utils/_expanded_weights/layer_norm_expanded_weights.py 2024-08-08T20:48:47.5987445Z inflating: build/lib/torch/nn/utils/_expanded_weights/linear_expanded_weights.py 2024-08-08T20:48:47.5988915Z inflating: build/lib/torch/nn/parameter.pyi 2024-08-08T20:48:47.5989770Z creating: build/lib/torch/onnx/ 2024-08-08T20:48:47.5990662Z inflating: build/lib/torch/onnx/__init__.py 2024-08-08T20:48:47.5991572Z inflating: build/lib/torch/onnx/_constants.py 2024-08-08T20:48:47.5992371Z inflating: build/lib/torch/onnx/_deprecation.py 2024-08-08T20:48:47.5993104Z inflating: build/lib/torch/onnx/_experimental.py 2024-08-08T20:48:47.5994154Z inflating: build/lib/torch/onnx/_exporter_states.py 2024-08-08T20:48:47.5995259Z inflating: build/lib/torch/onnx/_flags.py 2024-08-08T20:48:47.5996114Z inflating: build/lib/torch/onnx/_globals.py 2024-08-08T20:48:47.5996928Z inflating: build/lib/torch/onnx/_onnx_supported_ops.py 2024-08-08T20:48:47.5997849Z inflating: build/lib/torch/onnx/_type_utils.py 2024-08-08T20:48:47.5998489Z inflating: build/lib/torch/onnx/errors.py 2024-08-08T20:48:47.5999101Z inflating: build/lib/torch/onnx/operators.py 2024-08-08T20:48:47.5999783Z inflating: build/lib/torch/onnx/symbolic_caffe2.py 2024-08-08T20:48:47.6005313Z inflating: build/lib/torch/onnx/symbolic_helper.py 2024-08-08T20:48:47.6008745Z inflating: build/lib/torch/onnx/symbolic_opset10.py 2024-08-08T20:48:47.6013657Z inflating: build/lib/torch/onnx/symbolic_opset11.py 2024-08-08T20:48:47.6015811Z inflating: build/lib/torch/onnx/symbolic_opset12.py 2024-08-08T20:48:47.6019620Z inflating: build/lib/torch/onnx/symbolic_opset13.py 2024-08-08T20:48:47.6021181Z inflating: build/lib/torch/onnx/symbolic_opset14.py 2024-08-08T20:48:47.6022273Z inflating: build/lib/torch/onnx/symbolic_opset15.py 2024-08-08T20:48:47.6023624Z inflating: build/lib/torch/onnx/symbolic_opset16.py 2024-08-08T20:48:47.6025379Z inflating: build/lib/torch/onnx/symbolic_opset17.py 2024-08-08T20:48:47.6026884Z inflating: build/lib/torch/onnx/symbolic_opset18.py 2024-08-08T20:48:47.6028029Z inflating: build/lib/torch/onnx/symbolic_opset19.py 2024-08-08T20:48:47.6029249Z inflating: build/lib/torch/onnx/symbolic_opset20.py 2024-08-08T20:48:47.6030135Z inflating: build/lib/torch/onnx/symbolic_opset7.py 2024-08-08T20:48:47.6031758Z inflating: build/lib/torch/onnx/symbolic_opset8.py 2024-08-08T20:48:47.6049883Z inflating: build/lib/torch/onnx/symbolic_opset9.py 2024-08-08T20:48:47.6057687Z inflating: build/lib/torch/onnx/utils.py 2024-08-08T20:48:47.6064608Z inflating: build/lib/torch/onnx/verification.py 2024-08-08T20:48:47.6065539Z creating: build/lib/torch/onnx/_internal/ 2024-08-08T20:48:47.6066223Z extracting: build/lib/torch/onnx/_internal/__init__.py 2024-08-08T20:48:47.6072184Z inflating: build/lib/torch/onnx/_internal/_exporter_legacy.py 2024-08-08T20:48:47.6074596Z inflating: build/lib/torch/onnx/_internal/io_adapter.py 2024-08-08T20:48:47.6077220Z inflating: build/lib/torch/onnx/_internal/jit_utils.py 2024-08-08T20:48:47.6078980Z inflating: build/lib/torch/onnx/_internal/onnx_proto_utils.py 2024-08-08T20:48:47.6084529Z inflating: build/lib/torch/onnx/_internal/onnxruntime.py 2024-08-08T20:48:47.6086344Z inflating: build/lib/torch/onnx/_internal/registration.py 2024-08-08T20:48:47.6087665Z creating: build/lib/torch/onnx/_internal/diagnostics/ 2024-08-08T20:48:47.6088677Z inflating: build/lib/torch/onnx/_internal/diagnostics/__init__.py 2024-08-08T20:48:47.6089857Z inflating: build/lib/torch/onnx/_internal/diagnostics/_diagnostic.py 2024-08-08T20:48:47.6092950Z inflating: build/lib/torch/onnx/_internal/diagnostics/_rules.py 2024-08-08T20:48:47.6094187Z creating: build/lib/torch/onnx/_internal/diagnostics/infra/ 2024-08-08T20:48:47.6095434Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/__init__.py 2024-08-08T20:48:47.6096470Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/_infra.py 2024-08-08T20:48:47.6097946Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/context.py 2024-08-08T20:48:47.6099312Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/decorator.py 2024-08-08T20:48:47.6101016Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/formatter.py 2024-08-08T20:48:47.6102446Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/utils.py 2024-08-08T20:48:47.6103781Z creating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/ 2024-08-08T20:48:47.6105151Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/__init__.py 2024-08-08T20:48:47.6106949Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_address.py 2024-08-08T20:48:47.6108581Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_artifact.py 2024-08-08T20:48:47.6110220Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_artifact_change.py 2024-08-08T20:48:47.6112170Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_artifact_content.py 2024-08-08T20:48:47.6114040Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_artifact_location.py 2024-08-08T20:48:47.6115779Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_attachment.py 2024-08-08T20:48:47.6117397Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_code_flow.py 2024-08-08T20:48:47.6119408Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_configuration_override.py 2024-08-08T20:48:47.6121292Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_conversion.py 2024-08-08T20:48:47.6123150Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_edge.py 2024-08-08T20:48:47.6124964Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_edge_traversal.py 2024-08-08T20:48:47.6126752Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_exception.py 2024-08-08T20:48:47.6128482Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_external_properties.py 2024-08-08T20:48:47.6130493Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_external_property_file_reference.py 2024-08-08T20:48:47.6132693Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_external_property_file_references.py 2024-08-08T20:48:47.6134577Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_fix.py 2024-08-08T20:48:47.6136291Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_graph.py 2024-08-08T20:48:47.6137927Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_graph_traversal.py 2024-08-08T20:48:47.6139616Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_invocation.py 2024-08-08T20:48:47.6141362Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_location.py 2024-08-08T20:48:47.6142955Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_location_relationship.py 2024-08-08T20:48:47.6144858Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_logical_location.py 2024-08-08T20:48:47.6146617Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_message.py 2024-08-08T20:48:47.6148477Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_multiformat_message_string.py 2024-08-08T20:48:47.6150231Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_node.py 2024-08-08T20:48:47.6151787Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_notification.py 2024-08-08T20:48:47.6153469Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_physical_location.py 2024-08-08T20:48:47.6155217Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_property_bag.py 2024-08-08T20:48:47.6156916Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_rectangle.py 2024-08-08T20:48:47.6158847Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_region.py 2024-08-08T20:48:47.6160475Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_replacement.py 2024-08-08T20:48:47.6161897Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_reporting_configuration.py 2024-08-08T20:48:47.6163555Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_reporting_descriptor.py 2024-08-08T20:48:47.6165574Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_reporting_descriptor_reference.py 2024-08-08T20:48:47.6167394Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_reporting_descriptor_relationship.py 2024-08-08T20:48:47.6169055Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_result.py 2024-08-08T20:48:47.6170221Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_result_provenance.py 2024-08-08T20:48:47.6171327Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_run.py 2024-08-08T20:48:47.6172712Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_run_automation_details.py 2024-08-08T20:48:47.6174082Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_sarif_log.py 2024-08-08T20:48:47.6175596Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_special_locations.py 2024-08-08T20:48:47.6176710Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_stack.py 2024-08-08T20:48:47.6177792Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_stack_frame.py 2024-08-08T20:48:47.6179297Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_suppression.py 2024-08-08T20:48:47.6180948Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_thread_flow.py 2024-08-08T20:48:47.6182464Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_thread_flow_location.py 2024-08-08T20:48:47.6184083Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_tool.py 2024-08-08T20:48:47.6185327Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_tool_component.py 2024-08-08T20:48:47.6187109Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_tool_component_reference.py 2024-08-08T20:48:47.6188805Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_translation_metadata.py 2024-08-08T20:48:47.6190705Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_version_control_details.py 2024-08-08T20:48:47.6192491Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_web_request.py 2024-08-08T20:48:47.6193816Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/_web_response.py 2024-08-08T20:48:47.6195657Z inflating: build/lib/torch/onnx/_internal/diagnostics/infra/sarif/version.py 2024-08-08T20:48:47.6196812Z creating: build/lib/torch/onnx/_internal/fx/ 2024-08-08T20:48:47.6197788Z inflating: build/lib/torch/onnx/_internal/fx/__init__.py 2024-08-08T20:48:47.6198584Z inflating: build/lib/torch/onnx/_internal/fx/_pass.py 2024-08-08T20:48:47.6199430Z inflating: build/lib/torch/onnx/_internal/fx/decomposition_skip.py 2024-08-08T20:48:47.6200378Z inflating: build/lib/torch/onnx/_internal/fx/decomposition_table.py 2024-08-08T20:48:47.6201631Z inflating: build/lib/torch/onnx/_internal/fx/diagnostics.py 2024-08-08T20:48:47.6202928Z inflating: build/lib/torch/onnx/_internal/fx/dynamo_graph_extractor.py 2024-08-08T20:48:47.6203915Z inflating: build/lib/torch/onnx/_internal/fx/fx_onnx_interpreter.py 2024-08-08T20:48:47.6204896Z inflating: build/lib/torch/onnx/_internal/fx/fx_symbolic_graph_extractor.py 2024-08-08T20:48:47.6205947Z inflating: build/lib/torch/onnx/_internal/fx/onnxfunction_dispatcher.py 2024-08-08T20:48:47.6206895Z inflating: build/lib/torch/onnx/_internal/fx/op_validation.py 2024-08-08T20:48:47.6207928Z inflating: build/lib/torch/onnx/_internal/fx/patcher.py 2024-08-08T20:48:47.6209182Z inflating: build/lib/torch/onnx/_internal/fx/registration.py 2024-08-08T20:48:47.6210462Z inflating: build/lib/torch/onnx/_internal/fx/serialization.py 2024-08-08T20:48:47.6211605Z inflating: build/lib/torch/onnx/_internal/fx/torch_export_graph_extractor.py 2024-08-08T20:48:47.6212602Z inflating: build/lib/torch/onnx/_internal/fx/type_utils.py 2024-08-08T20:48:47.6213814Z creating: build/lib/torch/onnx/_internal/fx/analysis/ 2024-08-08T20:48:47.6214897Z inflating: build/lib/torch/onnx/_internal/fx/analysis/__init__.py 2024-08-08T20:48:47.6215960Z inflating: build/lib/torch/onnx/_internal/fx/analysis/unsupported_nodes.py 2024-08-08T20:48:47.6217134Z creating: build/lib/torch/onnx/_internal/fx/passes/ 2024-08-08T20:48:47.6217934Z inflating: build/lib/torch/onnx/_internal/fx/passes/__init__.py 2024-08-08T20:48:47.6219125Z inflating: build/lib/torch/onnx/_internal/fx/passes/_utils.py 2024-08-08T20:48:47.6220046Z inflating: build/lib/torch/onnx/_internal/fx/passes/decomp.py 2024-08-08T20:48:47.6220998Z inflating: build/lib/torch/onnx/_internal/fx/passes/functionalization.py 2024-08-08T20:48:47.6222264Z inflating: build/lib/torch/onnx/_internal/fx/passes/modularization.py 2024-08-08T20:48:47.6223335Z inflating: build/lib/torch/onnx/_internal/fx/passes/readability.py 2024-08-08T20:48:47.6224614Z inflating: build/lib/torch/onnx/_internal/fx/passes/type_promotion.py 2024-08-08T20:48:47.6225654Z inflating: build/lib/torch/onnx/_internal/fx/passes/virtualization.py 2024-08-08T20:48:47.6226400Z creating: build/lib/torch/optim/ 2024-08-08T20:48:47.6226974Z inflating: build/lib/torch/optim/__init__.py 2024-08-08T20:48:47.6228161Z inflating: build/lib/torch/optim/_adafactor.py 2024-08-08T20:48:47.6228833Z inflating: build/lib/torch/optim/_functional.py 2024-08-08T20:48:47.6229643Z inflating: build/lib/torch/optim/adadelta.py 2024-08-08T20:48:47.6230536Z inflating: build/lib/torch/optim/adagrad.py 2024-08-08T20:48:47.6231153Z inflating: build/lib/torch/optim/adam.py 2024-08-08T20:48:47.6231783Z inflating: build/lib/torch/optim/adamax.py 2024-08-08T20:48:47.6232864Z inflating: build/lib/torch/optim/adamw.py 2024-08-08T20:48:47.6233463Z inflating: build/lib/torch/optim/asgd.py 2024-08-08T20:48:47.6234081Z inflating: build/lib/torch/optim/lbfgs.py 2024-08-08T20:48:47.6240200Z inflating: build/lib/torch/optim/lr_scheduler.py 2024-08-08T20:48:47.6242984Z inflating: build/lib/torch/optim/nadam.py 2024-08-08T20:48:47.6247493Z inflating: build/lib/torch/optim/optimizer.py 2024-08-08T20:48:47.6250175Z inflating: build/lib/torch/optim/radam.py 2024-08-08T20:48:47.6252657Z inflating: build/lib/torch/optim/rmsprop.py 2024-08-08T20:48:47.6255053Z inflating: build/lib/torch/optim/rprop.py 2024-08-08T20:48:47.6257381Z inflating: build/lib/torch/optim/sgd.py 2024-08-08T20:48:47.6259370Z inflating: build/lib/torch/optim/sparse_adam.py 2024-08-08T20:48:47.6261935Z inflating: build/lib/torch/optim/swa_utils.py 2024-08-08T20:48:47.6263006Z creating: build/lib/torch/optim/_multi_tensor/ 2024-08-08T20:48:47.6264229Z inflating: build/lib/torch/optim/_multi_tensor/__init__.py 2024-08-08T20:48:47.6265439Z inflating: build/lib/torch/optim/_multi_tensor/__init__.pyi 2024-08-08T20:48:47.6265626Z creating: build/lib/torch/package/ 2024-08-08T20:48:47.6266003Z inflating: build/lib/torch/package/__init__.py 2024-08-08T20:48:47.6266569Z inflating: build/lib/torch/package/_digraph.py 2024-08-08T20:48:47.6267449Z inflating: build/lib/torch/package/_directory_reader.py 2024-08-08T20:48:47.6268504Z inflating: build/lib/torch/package/_importlib.py 2024-08-08T20:48:47.6269417Z inflating: build/lib/torch/package/_mangling.py 2024-08-08T20:48:47.6270397Z inflating: build/lib/torch/package/_mock.py 2024-08-08T20:48:47.6271750Z inflating: build/lib/torch/package/_package_pickler.py 2024-08-08T20:48:47.6272385Z inflating: build/lib/torch/package/_package_unpickler.py 2024-08-08T20:48:47.6274017Z inflating: build/lib/torch/package/_stdlib.py 2024-08-08T20:48:47.6275641Z inflating: build/lib/torch/package/file_structure_representation.py 2024-08-08T20:48:47.6276351Z inflating: build/lib/torch/package/find_file_dependencies.py 2024-08-08T20:48:47.6277546Z inflating: build/lib/torch/package/glob_group.py 2024-08-08T20:48:47.6279420Z inflating: build/lib/torch/package/importer.py 2024-08-08T20:48:47.6284513Z inflating: build/lib/torch/package/package_exporter.py 2024-08-08T20:48:47.6288076Z inflating: build/lib/torch/package/package_importer.py 2024-08-08T20:48:47.6288297Z creating: build/lib/torch/package/analyze/ 2024-08-08T20:48:47.6288960Z inflating: build/lib/torch/package/analyze/__init__.py 2024-08-08T20:48:47.6290112Z inflating: build/lib/torch/package/analyze/find_first_use_of_broken_modules.py 2024-08-08T20:48:47.6290733Z inflating: build/lib/torch/package/analyze/is_from_package.py 2024-08-08T20:48:47.6291875Z inflating: build/lib/torch/package/analyze/trace_dependencies.py 2024-08-08T20:48:47.6292443Z creating: build/lib/torch/profiler/ 2024-08-08T20:48:47.6293450Z inflating: build/lib/torch/profiler/__init__.py 2024-08-08T20:48:47.6298596Z inflating: build/lib/torch/profiler/_memory_profiler.py 2024-08-08T20:48:47.6301462Z inflating: build/lib/torch/profiler/_pattern_matcher.py 2024-08-08T20:48:47.6303561Z inflating: build/lib/torch/profiler/_utils.py 2024-08-08T20:48:47.6304155Z inflating: build/lib/torch/profiler/itt.py 2024-08-08T20:48:47.6308343Z inflating: build/lib/torch/profiler/profiler.py 2024-08-08T20:48:47.6308699Z inflating: build/lib/torch/profiler/python_tracer.py 2024-08-08T20:48:47.6309160Z creating: build/lib/torch/quantization/ 2024-08-08T20:48:47.6310072Z inflating: build/lib/torch/quantization/__init__.py 2024-08-08T20:48:47.6310967Z inflating: build/lib/torch/quantization/_numeric_suite.py 2024-08-08T20:48:47.6311833Z inflating: build/lib/torch/quantization/_numeric_suite_fx.py 2024-08-08T20:48:47.6313349Z inflating: build/lib/torch/quantization/_quantized_conversions.py 2024-08-08T20:48:47.6314087Z inflating: build/lib/torch/quantization/fake_quantize.py 2024-08-08T20:48:47.6315093Z inflating: build/lib/torch/quantization/fuse_modules.py 2024-08-08T20:48:47.6315653Z inflating: build/lib/torch/quantization/fuser_method_mappings.py 2024-08-08T20:48:47.6316679Z inflating: build/lib/torch/quantization/observer.py 2024-08-08T20:48:47.6317466Z inflating: build/lib/torch/quantization/qconfig.py 2024-08-08T20:48:47.6318315Z inflating: build/lib/torch/quantization/quant_type.py 2024-08-08T20:48:47.6319369Z inflating: build/lib/torch/quantization/quantization_mappings.py 2024-08-08T20:48:47.6320020Z inflating: build/lib/torch/quantization/quantize.py 2024-08-08T20:48:47.6321167Z inflating: build/lib/torch/quantization/quantize_fx.py 2024-08-08T20:48:47.6322055Z inflating: build/lib/torch/quantization/quantize_jit.py 2024-08-08T20:48:47.6322967Z inflating: build/lib/torch/quantization/stubs.py 2024-08-08T20:48:47.6323498Z inflating: build/lib/torch/quantization/utils.py 2024-08-08T20:48:47.6323952Z creating: build/lib/torch/quantization/fx/ 2024-08-08T20:48:47.6325135Z inflating: build/lib/torch/quantization/fx/__init__.py 2024-08-08T20:48:47.6326104Z inflating: build/lib/torch/quantization/fx/_equalize.py 2024-08-08T20:48:47.6326861Z inflating: build/lib/torch/quantization/fx/convert.py 2024-08-08T20:48:47.6327579Z inflating: build/lib/torch/quantization/fx/fuse.py 2024-08-08T20:48:47.6328209Z inflating: build/lib/torch/quantization/fx/fusion_patterns.py 2024-08-08T20:48:47.6329037Z inflating: build/lib/torch/quantization/fx/graph_module.py 2024-08-08T20:48:47.6330014Z inflating: build/lib/torch/quantization/fx/match_utils.py 2024-08-08T20:48:47.6330971Z inflating: build/lib/torch/quantization/fx/pattern_utils.py 2024-08-08T20:48:47.6405883Z inflating: build/lib/torch/quantization/fx/prepare.py 2024-08-08T20:48:47.6407063Z inflating: build/lib/torch/quantization/fx/quantization_patterns.py 2024-08-08T20:48:47.6407770Z inflating: build/lib/torch/quantization/fx/quantization_types.py 2024-08-08T20:48:47.6408339Z inflating: build/lib/torch/quantization/fx/utils.py 2024-08-08T20:48:47.6408598Z creating: build/lib/torch/signal/ 2024-08-08T20:48:47.6408838Z inflating: build/lib/torch/signal/__init__.py 2024-08-08T20:48:47.6409179Z creating: build/lib/torch/signal/windows/ 2024-08-08T20:48:47.6409465Z inflating: build/lib/torch/signal/windows/__init__.py 2024-08-08T20:48:47.6409748Z inflating: build/lib/torch/signal/windows/windows.py 2024-08-08T20:48:47.6409942Z creating: build/lib/torch/sparse/ 2024-08-08T20:48:47.6410194Z inflating: build/lib/torch/sparse/__init__.py 2024-08-08T20:48:47.6410572Z inflating: build/lib/torch/sparse/_semi_structured_conversions.py 2024-08-08T20:48:47.6410879Z inflating: build/lib/torch/sparse/_semi_structured_ops.py 2024-08-08T20:48:47.6411150Z inflating: build/lib/torch/sparse/_triton_ops.py 2024-08-08T20:48:47.6411413Z inflating: build/lib/torch/sparse/_triton_ops_meta.py 2024-08-08T20:48:47.6411675Z inflating: build/lib/torch/sparse/semi_structured.py 2024-08-08T20:48:47.6411848Z creating: build/lib/torch/special/ 2024-08-08T20:48:47.6412072Z inflating: build/lib/torch/special/__init__.py 2024-08-08T20:48:47.6412233Z creating: build/lib/torch/testing/ 2024-08-08T20:48:47.6412457Z inflating: build/lib/torch/testing/__init__.py 2024-08-08T20:48:47.6412703Z inflating: build/lib/torch/testing/_comparison.py 2024-08-08T20:48:47.6412932Z inflating: build/lib/torch/testing/_creation.py 2024-08-08T20:48:47.6413408Z inflating: build/lib/torch/testing/_utils.py 2024-08-08T20:48:47.6413622Z creating: build/lib/torch/testing/_internal/ 2024-08-08T20:48:47.6414206Z extracting: build/lib/torch/testing/_internal/__init__.py 2024-08-08T20:48:47.6414740Z inflating: build/lib/torch/testing/_internal/autocast_test_lists.py 2024-08-08T20:48:47.6415349Z inflating: build/lib/torch/testing/_internal/autograd_function_db.py 2024-08-08T20:48:47.6415797Z inflating: build/lib/torch/testing/_internal/check_kernel_launches.py 2024-08-08T20:48:47.6416205Z inflating: build/lib/torch/testing/_internal/common_cuda.py 2024-08-08T20:48:47.6416639Z inflating: build/lib/torch/testing/_internal/common_device_type.py 2024-08-08T20:48:47.6417045Z inflating: build/lib/torch/testing/_internal/common_dist_composable.py 2024-08-08T20:48:47.6417402Z inflating: build/lib/torch/testing/_internal/common_distributed.py 2024-08-08T20:48:47.6417723Z inflating: build/lib/torch/testing/_internal/common_dtype.py 2024-08-08T20:48:47.6422097Z inflating: build/lib/torch/testing/_internal/common_fsdp.py 2024-08-08T20:48:47.6424537Z inflating: build/lib/torch/testing/_internal/common_jit.py 2024-08-08T20:48:47.6503531Z inflating: build/lib/torch/testing/_internal/common_methods_invocations.py 2024-08-08T20:48:47.6504081Z inflating: build/lib/torch/testing/_internal/common_mkldnn.py 2024-08-08T20:48:47.6517555Z inflating: build/lib/torch/testing/_internal/common_modules.py 2024-08-08T20:48:47.6529303Z inflating: build/lib/torch/testing/_internal/common_nn.py 2024-08-08T20:48:47.6535351Z inflating: build/lib/torch/testing/_internal/common_optimizers.py 2024-08-08T20:48:47.6537114Z inflating: build/lib/torch/testing/_internal/common_pruning.py 2024-08-08T20:48:47.6545879Z inflating: build/lib/torch/testing/_internal/common_quantization.py 2024-08-08T20:48:47.6547547Z inflating: build/lib/torch/testing/_internal/common_quantized.py 2024-08-08T20:48:47.6548668Z inflating: build/lib/torch/testing/_internal/common_subclass.py 2024-08-08T20:48:47.6569298Z inflating: build/lib/torch/testing/_internal/common_utils.py 2024-08-08T20:48:47.6572268Z inflating: build/lib/torch/testing/_internal/composite_compliance.py 2024-08-08T20:48:47.6574524Z inflating: build/lib/torch/testing/_internal/custom_op_db.py 2024-08-08T20:48:47.6575274Z inflating: build/lib/torch/testing/_internal/custom_tensor.py 2024-08-08T20:48:47.6577077Z inflating: build/lib/torch/testing/_internal/dist_utils.py 2024-08-08T20:48:47.6578206Z inflating: build/lib/torch/testing/_internal/dynamo_test_failures.py 2024-08-08T20:48:47.6579746Z inflating: build/lib/torch/testing/_internal/hop_db.py 2024-08-08T20:48:47.6581907Z inflating: build/lib/torch/testing/_internal/hypothesis_utils.py 2024-08-08T20:48:47.6582723Z inflating: build/lib/torch/testing/_internal/inductor_utils.py 2024-08-08T20:48:47.6586535Z inflating: build/lib/torch/testing/_internal/jit_metaprogramming_utils.py 2024-08-08T20:48:47.6590112Z inflating: build/lib/torch/testing/_internal/jit_utils.py 2024-08-08T20:48:47.6591636Z inflating: build/lib/torch/testing/_internal/logging_tensor.py 2024-08-08T20:48:47.6593051Z inflating: build/lib/torch/testing/_internal/logging_utils.py 2024-08-08T20:48:47.6593773Z inflating: build/lib/torch/testing/_internal/quantization_torch_package_models.py 2024-08-08T20:48:47.6594632Z inflating: build/lib/torch/testing/_internal/static_module.py 2024-08-08T20:48:47.6596022Z inflating: build/lib/torch/testing/_internal/torchbind_impls.py 2024-08-08T20:48:47.6597611Z inflating: build/lib/torch/testing/_internal/triton_utils.py 2024-08-08T20:48:47.6598771Z inflating: build/lib/torch/testing/_internal/two_tensor.py 2024-08-08T20:48:47.6599199Z creating: build/lib/torch/testing/_internal/codegen/ 2024-08-08T20:48:47.6600169Z extracting: build/lib/torch/testing/_internal/codegen/__init__.py 2024-08-08T20:48:47.6600996Z creating: build/lib/torch/testing/_internal/data/ 2024-08-08T20:48:47.6601644Z extracting: build/lib/torch/testing/_internal/data/__init__.py 2024-08-08T20:48:47.6602164Z inflating: build/lib/torch/testing/_internal/data/network1.py 2024-08-08T20:48:47.6603230Z inflating: build/lib/torch/testing/_internal/data/network2.py 2024-08-08T20:48:47.6603715Z creating: build/lib/torch/testing/_internal/distributed/ 2024-08-08T20:48:47.6604282Z extracting: build/lib/torch/testing/_internal/distributed/__init__.py 2024-08-08T20:48:47.6605386Z inflating: build/lib/torch/testing/_internal/distributed/checkpoint_utils.py 2024-08-08T20:48:47.6606668Z inflating: build/lib/torch/testing/_internal/distributed/common_state_dict.py 2024-08-08T20:48:47.6610025Z inflating: build/lib/torch/testing/_internal/distributed/ddp_under_dist_autograd_test.py 2024-08-08T20:48:47.6641010Z inflating: build/lib/torch/testing/_internal/distributed/distributed_test.py 2024-08-08T20:48:47.6641795Z inflating: build/lib/torch/testing/_internal/distributed/distributed_utils.py 2024-08-08T20:48:47.6642420Z inflating: build/lib/torch/testing/_internal/distributed/fake_pg.py 2024-08-08T20:48:47.6645708Z inflating: build/lib/torch/testing/_internal/distributed/multi_threaded_pg.py 2024-08-08T20:48:47.6646710Z inflating: build/lib/torch/testing/_internal/distributed/rpc_utils.py 2024-08-08T20:48:47.6647561Z creating: build/lib/torch/testing/_internal/distributed/_shard/ 2024-08-08T20:48:47.6648233Z extracting: build/lib/torch/testing/_internal/distributed/_shard/__init__.py 2024-08-08T20:48:47.6649027Z inflating: build/lib/torch/testing/_internal/distributed/_shard/test_common.py 2024-08-08T20:48:47.6649901Z creating: build/lib/torch/testing/_internal/distributed/_shard/sharded_tensor/ 2024-08-08T20:48:47.6651190Z inflating: build/lib/torch/testing/_internal/distributed/_shard/sharded_tensor/__init__.py 2024-08-08T20:48:47.6652596Z inflating: build/lib/torch/testing/_internal/distributed/_shard/sharded_tensor/_test_ops_common.py 2024-08-08T20:48:47.6653433Z inflating: build/lib/torch/testing/_internal/distributed/_shard/sharded_tensor/_test_st_common.py 2024-08-08T20:48:47.6653988Z creating: build/lib/torch/testing/_internal/distributed/_tensor/ 2024-08-08T20:48:47.6654367Z extracting: build/lib/torch/testing/_internal/distributed/_tensor/__init__.py 2024-08-08T20:48:47.6656989Z inflating: build/lib/torch/testing/_internal/distributed/_tensor/common_dtensor.py 2024-08-08T20:48:47.6657483Z creating: build/lib/torch/testing/_internal/distributed/nn/ 2024-08-08T20:48:47.6658075Z extracting: build/lib/torch/testing/_internal/distributed/nn/__init__.py 2024-08-08T20:48:47.6658819Z creating: build/lib/torch/testing/_internal/distributed/nn/api/ 2024-08-08T20:48:47.6659298Z extracting: build/lib/torch/testing/_internal/distributed/nn/api/__init__.py 2024-08-08T20:48:47.6662245Z inflating: build/lib/torch/testing/_internal/distributed/nn/api/remote_module_test.py 2024-08-08T20:48:47.6662772Z creating: build/lib/torch/testing/_internal/distributed/rpc/ 2024-08-08T20:48:47.6663175Z extracting: build/lib/torch/testing/_internal/distributed/rpc/__init__.py 2024-08-08T20:48:47.6671622Z inflating: build/lib/torch/testing/_internal/distributed/rpc/dist_autograd_test.py 2024-08-08T20:48:47.6673244Z inflating: build/lib/torch/testing/_internal/distributed/rpc/dist_optimizer_test.py 2024-08-08T20:48:47.6675093Z inflating: build/lib/torch/testing/_internal/distributed/rpc/faulty_agent_rpc_test.py 2024-08-08T20:48:47.6676307Z inflating: build/lib/torch/testing/_internal/distributed/rpc/faulty_rpc_agent_test_fixture.py 2024-08-08T20:48:47.6677322Z inflating: build/lib/torch/testing/_internal/distributed/rpc/rpc_agent_test_fixture.py 2024-08-08T20:48:47.6695112Z inflating: build/lib/torch/testing/_internal/distributed/rpc/rpc_test.py 2024-08-08T20:48:47.6696076Z inflating: build/lib/torch/testing/_internal/distributed/rpc/tensorpipe_rpc_agent_test_fixture.py 2024-08-08T20:48:47.6696909Z creating: build/lib/torch/testing/_internal/distributed/rpc/examples/ 2024-08-08T20:48:47.6697508Z extracting: build/lib/torch/testing/_internal/distributed/rpc/examples/__init__.py 2024-08-08T20:48:47.6698720Z inflating: build/lib/torch/testing/_internal/distributed/rpc/examples/parameter_server_test.py 2024-08-08T20:48:47.6700935Z inflating: build/lib/torch/testing/_internal/distributed/rpc/examples/reinforcement_learning_rpc_test.py 2024-08-08T20:48:47.6701504Z creating: build/lib/torch/testing/_internal/distributed/rpc/jit/ 2024-08-08T20:48:47.6701866Z extracting: build/lib/torch/testing/_internal/distributed/rpc/jit/__init__.py 2024-08-08T20:48:47.6702878Z inflating: build/lib/torch/testing/_internal/distributed/rpc/jit/dist_autograd_test.py 2024-08-08T20:48:47.6707407Z inflating: build/lib/torch/testing/_internal/distributed/rpc/jit/rpc_test.py 2024-08-08T20:48:47.6708705Z inflating: build/lib/torch/testing/_internal/distributed/rpc/jit/rpc_test_faulty.py 2024-08-08T20:48:47.6709211Z creating: build/lib/torch/testing/_internal/generated/ 2024-08-08T20:48:47.6710012Z extracting: build/lib/torch/testing/_internal/generated/__init__.py 2024-08-08T20:48:47.6737512Z inflating: build/lib/torch/testing/_internal/generated/annotated_fn_args.py 2024-08-08T20:48:47.6737986Z creating: build/lib/torch/testing/_internal/opinfo/ 2024-08-08T20:48:47.6738386Z inflating: build/lib/torch/testing/_internal/opinfo/__init__.py 2024-08-08T20:48:47.6748512Z inflating: build/lib/torch/testing/_internal/opinfo/core.py 2024-08-08T20:48:47.6749845Z inflating: build/lib/torch/testing/_internal/opinfo/refs.py 2024-08-08T20:48:47.6751800Z inflating: build/lib/torch/testing/_internal/opinfo/utils.py 2024-08-08T20:48:47.6752375Z creating: build/lib/torch/testing/_internal/opinfo/definitions/ 2024-08-08T20:48:47.6752998Z inflating: build/lib/torch/testing/_internal/opinfo/definitions/__init__.py 2024-08-08T20:48:47.6756873Z inflating: build/lib/torch/testing/_internal/opinfo/definitions/_masked.py 2024-08-08T20:48:47.6759723Z inflating: build/lib/torch/testing/_internal/opinfo/definitions/fft.py 2024-08-08T20:48:47.6766709Z inflating: build/lib/torch/testing/_internal/opinfo/definitions/linalg.py 2024-08-08T20:48:47.6768498Z inflating: build/lib/torch/testing/_internal/opinfo/definitions/nested.py 2024-08-08T20:48:47.6770400Z inflating: build/lib/torch/testing/_internal/opinfo/definitions/signal.py 2024-08-08T20:48:47.6773527Z inflating: build/lib/torch/testing/_internal/opinfo/definitions/sparse.py 2024-08-08T20:48:47.6776285Z inflating: build/lib/torch/testing/_internal/opinfo/definitions/special.py 2024-08-08T20:48:47.6776860Z creating: build/lib/torch/testing/_internal/optests/ 2024-08-08T20:48:47.6777409Z inflating: build/lib/torch/testing/_internal/optests/__init__.py 2024-08-08T20:48:47.6778942Z inflating: build/lib/torch/testing/_internal/optests/aot_autograd.py 2024-08-08T20:48:47.6780616Z inflating: build/lib/torch/testing/_internal/optests/autograd_registration.py 2024-08-08T20:48:47.6781186Z inflating: build/lib/torch/testing/_internal/optests/fake_tensor.py 2024-08-08T20:48:47.6784873Z inflating: build/lib/torch/testing/_internal/optests/generate_tests.py 2024-08-08T20:48:47.6785896Z inflating: build/lib/torch/testing/_internal/optests/make_fx.py 2024-08-08T20:48:47.6786408Z creating: build/lib/torch/testing/_internal/test_module/ 2024-08-08T20:48:47.6786816Z extracting: build/lib/torch/testing/_internal/test_module/__init__.py 2024-08-08T20:48:47.6788554Z inflating: build/lib/torch/testing/_internal/test_module/future_div.py 2024-08-08T20:48:47.6789276Z inflating: build/lib/torch/testing/_internal/test_module/no_future_div.py 2024-08-08T20:48:47.6789594Z creating: build/lib/torch/utils/ 2024-08-08T20:48:47.6790752Z inflating: build/lib/torch/utils/__init__.py 2024-08-08T20:48:47.6792073Z inflating: build/lib/torch/utils/_backport_slots.py 2024-08-08T20:48:47.6794319Z inflating: build/lib/torch/utils/_config_module.py 2024-08-08T20:48:47.6796016Z inflating: build/lib/torch/utils/_content_store.py 2024-08-08T20:48:47.6797186Z inflating: build/lib/torch/utils/_contextlib.py 2024-08-08T20:48:47.6798164Z inflating: build/lib/torch/utils/_cpp_extension_versioner.py 2024-08-08T20:48:47.6801609Z inflating: build/lib/torch/utils/_cxx_pytree.py 2024-08-08T20:48:47.6802763Z inflating: build/lib/torch/utils/_device.py 2024-08-08T20:48:47.6803689Z inflating: build/lib/torch/utils/_exposed_in.py 2024-08-08T20:48:47.6804625Z inflating: build/lib/torch/utils/_foreach_utils.py 2024-08-08T20:48:47.6807274Z inflating: build/lib/torch/utils/_freeze.py 2024-08-08T20:48:47.6808533Z inflating: build/lib/torch/utils/_get_clean_triton.py 2024-08-08T20:48:47.6809425Z inflating: build/lib/torch/utils/_import_utils.py 2024-08-08T20:48:47.6809955Z inflating: build/lib/torch/utils/_mode_utils.py 2024-08-08T20:48:47.6811626Z inflating: build/lib/torch/utils/_ordered_set.py 2024-08-08T20:48:47.6815025Z inflating: build/lib/torch/utils/_python_dispatch.py 2024-08-08T20:48:47.6819961Z inflating: build/lib/torch/utils/_pytree.py 2024-08-08T20:48:47.6820607Z inflating: build/lib/torch/utils/_stats.py 2024-08-08T20:48:47.6822529Z inflating: build/lib/torch/utils/_traceback.py 2024-08-08T20:48:47.6823163Z inflating: build/lib/torch/utils/_triton.py 2024-08-08T20:48:47.6824032Z inflating: build/lib/torch/utils/_typing_utils.py 2024-08-08T20:48:47.6825553Z inflating: build/lib/torch/utils/_zip.py 2024-08-08T20:48:47.6828079Z inflating: build/lib/torch/utils/backend_registration.py 2024-08-08T20:48:47.6830677Z inflating: build/lib/torch/utils/bundled_inputs.py 2024-08-08T20:48:47.6837344Z inflating: build/lib/torch/utils/checkpoint.py 2024-08-08T20:48:47.6840440Z inflating: build/lib/torch/utils/collect_env.py 2024-08-08T20:48:47.6841383Z inflating: build/lib/torch/utils/cpp_backtrace.py 2024-08-08T20:48:47.6851838Z inflating: build/lib/torch/utils/cpp_extension.py 2024-08-08T20:48:47.6852295Z inflating: build/lib/torch/utils/deterministic.py 2024-08-08T20:48:47.6854015Z inflating: build/lib/torch/utils/dlpack.py 2024-08-08T20:48:47.6854467Z inflating: build/lib/torch/utils/file_baton.py 2024-08-08T20:48:47.6857792Z inflating: build/lib/torch/utils/flop_counter.py 2024-08-08T20:48:47.6862011Z inflating: build/lib/torch/utils/hooks.py 2024-08-08T20:48:47.6863320Z inflating: build/lib/torch/utils/mkldnn.py 2024-08-08T20:48:47.6864736Z inflating: build/lib/torch/utils/mobile_optimizer.py 2024-08-08T20:48:47.6865137Z inflating: build/lib/torch/utils/model_zoo.py 2024-08-08T20:48:47.6866726Z inflating: build/lib/torch/utils/module_tracker.py 2024-08-08T20:48:47.6867996Z inflating: build/lib/torch/utils/show_pickle.py 2024-08-08T20:48:47.6869477Z inflating: build/lib/torch/utils/throughput_benchmark.py 2024-08-08T20:48:47.6871223Z inflating: build/lib/torch/utils/weak.py 2024-08-08T20:48:47.6871634Z creating: build/lib/torch/utils/_strobelight/ 2024-08-08T20:48:47.6872387Z extracting: build/lib/torch/utils/_strobelight/__init__.py 2024-08-08T20:48:47.6874587Z inflating: build/lib/torch/utils/_strobelight/cli_function_profiler.py 2024-08-08T20:48:47.6874992Z creating: build/lib/torch/utils/_sympy/ 2024-08-08T20:48:47.6875286Z extracting: build/lib/torch/utils/_sympy/__init__.py 2024-08-08T20:48:47.6878199Z inflating: build/lib/torch/utils/_sympy/functions.py 2024-08-08T20:48:47.6879491Z inflating: build/lib/torch/utils/_sympy/interp.py 2024-08-08T20:48:47.6881235Z inflating: build/lib/torch/utils/_sympy/numbers.py 2024-08-08T20:48:47.6882372Z inflating: build/lib/torch/utils/_sympy/reference.py 2024-08-08T20:48:47.6883435Z inflating: build/lib/torch/utils/_sympy/singleton_int.py 2024-08-08T20:48:47.6884770Z inflating: build/lib/torch/utils/_sympy/solve.py 2024-08-08T20:48:47.6885953Z inflating: build/lib/torch/utils/_sympy/symbol.py 2024-08-08T20:48:47.6890134Z inflating: build/lib/torch/utils/_sympy/value_ranges.py 2024-08-08T20:48:47.6890534Z creating: build/lib/torch/utils/backcompat/ 2024-08-08T20:48:47.6891364Z inflating: build/lib/torch/utils/backcompat/__init__.py 2024-08-08T20:48:47.6891711Z creating: build/lib/torch/utils/benchmark/ 2024-08-08T20:48:47.6892715Z inflating: build/lib/torch/utils/benchmark/__init__.py 2024-08-08T20:48:47.6893186Z creating: build/lib/torch/utils/benchmark/examples/ 2024-08-08T20:48:47.6893742Z extracting: build/lib/torch/utils/benchmark/examples/__init__.py 2024-08-08T20:48:47.6895705Z inflating: build/lib/torch/utils/benchmark/examples/blas_compare_setup.py 2024-08-08T20:48:47.6896256Z inflating: build/lib/torch/utils/benchmark/examples/compare.py 2024-08-08T20:48:47.6897366Z inflating: build/lib/torch/utils/benchmark/examples/fuzzer.py 2024-08-08T20:48:47.6898765Z inflating: build/lib/torch/utils/benchmark/examples/op_benchmark.py 2024-08-08T20:48:47.6899448Z inflating: build/lib/torch/utils/benchmark/examples/simple_timeit.py 2024-08-08T20:48:47.6900951Z inflating: build/lib/torch/utils/benchmark/examples/spectral_ops_fuzz_test.py 2024-08-08T20:48:47.6901529Z creating: build/lib/torch/utils/benchmark/op_fuzzers/ 2024-08-08T20:48:47.6902041Z extracting: build/lib/torch/utils/benchmark/op_fuzzers/__init__.py 2024-08-08T20:48:47.6902890Z inflating: build/lib/torch/utils/benchmark/op_fuzzers/binary.py 2024-08-08T20:48:47.6905064Z inflating: build/lib/torch/utils/benchmark/op_fuzzers/sparse_binary.py 2024-08-08T20:48:47.6905765Z inflating: build/lib/torch/utils/benchmark/op_fuzzers/sparse_unary.py 2024-08-08T20:48:47.6907030Z inflating: build/lib/torch/utils/benchmark/op_fuzzers/spectral.py 2024-08-08T20:48:47.6908149Z inflating: build/lib/torch/utils/benchmark/op_fuzzers/unary.py 2024-08-08T20:48:47.6908578Z creating: build/lib/torch/utils/benchmark/utils/ 2024-08-08T20:48:47.6909119Z extracting: build/lib/torch/utils/benchmark/utils/__init__.py 2024-08-08T20:48:47.6909964Z inflating: build/lib/torch/utils/benchmark/utils/_stubs.py 2024-08-08T20:48:47.6912319Z inflating: build/lib/torch/utils/benchmark/utils/common.py 2024-08-08T20:48:47.6914265Z inflating: build/lib/torch/utils/benchmark/utils/compare.py 2024-08-08T20:48:47.6915641Z inflating: build/lib/torch/utils/benchmark/utils/compile.py 2024-08-08T20:48:47.6917107Z inflating: build/lib/torch/utils/benchmark/utils/cpp_jit.py 2024-08-08T20:48:47.6919627Z inflating: build/lib/torch/utils/benchmark/utils/fuzzer.py 2024-08-08T20:48:47.6920923Z inflating: build/lib/torch/utils/benchmark/utils/sparse_fuzzer.py 2024-08-08T20:48:47.6923960Z inflating: build/lib/torch/utils/benchmark/utils/timer.py 2024-08-08T20:48:47.6924552Z creating: build/lib/torch/utils/benchmark/utils/valgrind_wrapper/ 2024-08-08T20:48:47.6925235Z extracting: build/lib/torch/utils/benchmark/utils/valgrind_wrapper/__init__.py 2024-08-08T20:48:47.6929760Z inflating: build/lib/torch/utils/benchmark/utils/valgrind_wrapper/timer_interface.py 2024-08-08T20:48:47.6930524Z inflating: build/lib/torch/utils/benchmark/utils/valgrind_wrapper/compat_bindings.cpp 2024-08-08T20:48:47.6931621Z inflating: build/lib/torch/utils/benchmark/utils/valgrind_wrapper/timer_callgrind_template.cpp 2024-08-08T20:48:47.6933004Z inflating: build/lib/torch/utils/benchmark/utils/valgrind_wrapper/callgrind.h 2024-08-08T20:48:47.6957386Z inflating: build/lib/torch/utils/benchmark/utils/valgrind_wrapper/valgrind.h 2024-08-08T20:48:47.6958023Z inflating: build/lib/torch/utils/benchmark/utils/timeit_template.cpp 2024-08-08T20:48:47.6958773Z creating: build/lib/torch/utils/bottleneck/ 2024-08-08T20:48:47.6959291Z extracting: build/lib/torch/utils/bottleneck/__init__.py 2024-08-08T20:48:47.6960814Z inflating: build/lib/torch/utils/bottleneck/__main__.py 2024-08-08T20:48:47.6961192Z creating: build/lib/torch/utils/data/ 2024-08-08T20:48:47.6962170Z inflating: build/lib/torch/utils/data/__init__.py 2024-08-08T20:48:47.6963064Z inflating: build/lib/torch/utils/data/backward_compatibility.py 2024-08-08T20:48:47.6970732Z inflating: build/lib/torch/utils/data/dataloader.py 2024-08-08T20:48:47.6973236Z inflating: build/lib/torch/utils/data/dataset.py 2024-08-08T20:48:47.6974564Z inflating: build/lib/torch/utils/data/distributed.py 2024-08-08T20:48:47.6975829Z inflating: build/lib/torch/utils/data/graph.py 2024-08-08T20:48:47.6977155Z inflating: build/lib/torch/utils/data/graph_settings.py 2024-08-08T20:48:47.6979294Z inflating: build/lib/torch/utils/data/sampler.py 2024-08-08T20:48:47.6979572Z creating: build/lib/torch/utils/data/_utils/ 2024-08-08T20:48:47.6980470Z inflating: build/lib/torch/utils/data/_utils/__init__.py 2024-08-08T20:48:47.6982847Z inflating: build/lib/torch/utils/data/_utils/collate.py 2024-08-08T20:48:47.6983408Z inflating: build/lib/torch/utils/data/_utils/fetch.py 2024-08-08T20:48:47.6984470Z inflating: build/lib/torch/utils/data/_utils/pin_memory.py 2024-08-08T20:48:47.6985648Z inflating: build/lib/torch/utils/data/_utils/signal_handling.py 2024-08-08T20:48:47.6988051Z inflating: build/lib/torch/utils/data/_utils/worker.py 2024-08-08T20:48:47.6988654Z creating: build/lib/torch/utils/data/datapipes/ 2024-08-08T20:48:47.6989767Z inflating: build/lib/torch/utils/data/datapipes/__init__.py 2024-08-08T20:48:47.6991492Z inflating: build/lib/torch/utils/data/datapipes/_decorator.py 2024-08-08T20:48:47.6993345Z inflating: build/lib/torch/utils/data/datapipes/_hook_iterator.py 2024-08-08T20:48:47.6995528Z inflating: build/lib/torch/utils/data/datapipes/_typing.py 2024-08-08T20:48:47.6997677Z inflating: build/lib/torch/utils/data/datapipes/datapipe.py 2024-08-08T20:48:47.6999647Z inflating: build/lib/torch/utils/data/datapipes/gen_pyi.py 2024-08-08T20:48:47.7000177Z creating: build/lib/torch/utils/data/datapipes/dataframe/ 2024-08-08T20:48:47.7000782Z inflating: build/lib/torch/utils/data/datapipes/dataframe/__init__.py 2024-08-08T20:48:47.7002011Z inflating: build/lib/torch/utils/data/datapipes/dataframe/dataframe_wrapper.py 2024-08-08T20:48:47.7003961Z inflating: build/lib/torch/utils/data/datapipes/dataframe/dataframes.py 2024-08-08T20:48:47.7005143Z inflating: build/lib/torch/utils/data/datapipes/dataframe/datapipes.py 2024-08-08T20:48:47.7005873Z inflating: build/lib/torch/utils/data/datapipes/dataframe/structures.py 2024-08-08T20:48:47.7006299Z creating: build/lib/torch/utils/data/datapipes/iter/ 2024-08-08T20:48:47.7007259Z inflating: build/lib/torch/utils/data/datapipes/iter/__init__.py 2024-08-08T20:48:47.7009073Z inflating: build/lib/torch/utils/data/datapipes/iter/callable.py 2024-08-08T20:48:47.7010451Z inflating: build/lib/torch/utils/data/datapipes/iter/combinatorics.py 2024-08-08T20:48:47.7013542Z inflating: build/lib/torch/utils/data/datapipes/iter/combining.py 2024-08-08T20:48:47.7014590Z inflating: build/lib/torch/utils/data/datapipes/iter/filelister.py 2024-08-08T20:48:47.7015764Z inflating: build/lib/torch/utils/data/datapipes/iter/fileopener.py 2024-08-08T20:48:47.7017722Z inflating: build/lib/torch/utils/data/datapipes/iter/grouping.py 2024-08-08T20:48:47.7018694Z inflating: build/lib/torch/utils/data/datapipes/iter/routeddecoder.py 2024-08-08T20:48:47.7020113Z inflating: build/lib/torch/utils/data/datapipes/iter/selecting.py 2024-08-08T20:48:47.7021304Z inflating: build/lib/torch/utils/data/datapipes/iter/sharding.py 2024-08-08T20:48:47.7022384Z inflating: build/lib/torch/utils/data/datapipes/iter/streamreader.py 2024-08-08T20:48:47.7023374Z inflating: build/lib/torch/utils/data/datapipes/iter/utils.py 2024-08-08T20:48:47.7023815Z creating: build/lib/torch/utils/data/datapipes/map/ 2024-08-08T20:48:47.7024711Z inflating: build/lib/torch/utils/data/datapipes/map/__init__.py 2024-08-08T20:48:47.7025780Z inflating: build/lib/torch/utils/data/datapipes/map/callable.py 2024-08-08T20:48:47.7027138Z inflating: build/lib/torch/utils/data/datapipes/map/combinatorics.py 2024-08-08T20:48:47.7028197Z inflating: build/lib/torch/utils/data/datapipes/map/combining.py 2024-08-08T20:48:47.7029148Z inflating: build/lib/torch/utils/data/datapipes/map/grouping.py 2024-08-08T20:48:47.7031168Z inflating: build/lib/torch/utils/data/datapipes/map/utils.py 2024-08-08T20:48:47.7031520Z creating: build/lib/torch/utils/data/datapipes/utils/ 2024-08-08T20:48:47.7032036Z extracting: build/lib/torch/utils/data/datapipes/utils/__init__.py 2024-08-08T20:48:47.7034074Z inflating: build/lib/torch/utils/data/datapipes/utils/common.py 2024-08-08T20:48:47.7035970Z inflating: build/lib/torch/utils/data/datapipes/utils/decoder.py 2024-08-08T20:48:47.7037072Z inflating: build/lib/torch/utils/data/datapipes/utils/snapshot.py 2024-08-08T20:48:47.7037287Z creating: build/lib/torch/utils/hipify/ 2024-08-08T20:48:47.7038569Z inflating: build/lib/torch/utils/hipify/__init__.py 2024-08-08T20:48:47.7039553Z inflating: build/lib/torch/utils/hipify/constants.py 2024-08-08T20:48:47.7061907Z inflating: build/lib/torch/utils/hipify/cuda_to_hip_mappings.py 2024-08-08T20:48:47.7067279Z inflating: build/lib/torch/utils/hipify/hipify_python.py 2024-08-08T20:48:47.7067678Z extracting: build/lib/torch/utils/hipify/version.py 2024-08-08T20:48:47.7068027Z creating: build/lib/torch/utils/jit/ 2024-08-08T20:48:47.7068450Z extracting: build/lib/torch/utils/jit/__init__.py 2024-08-08T20:48:47.7069720Z inflating: build/lib/torch/utils/jit/log_extract.py 2024-08-08T20:48:47.7070124Z creating: build/lib/torch/utils/model_dump/ 2024-08-08T20:48:47.7072873Z inflating: build/lib/torch/utils/model_dump/__init__.py 2024-08-08T20:48:47.7073392Z inflating: build/lib/torch/utils/model_dump/__main__.py 2024-08-08T20:48:47.7074118Z inflating: build/lib/torch/utils/model_dump/skeleton.html 2024-08-08T20:48:47.7076687Z inflating: build/lib/torch/utils/model_dump/code.js 2024-08-08T20:48:47.7077213Z inflating: build/lib/torch/utils/model_dump/htm.mjs 2024-08-08T20:48:47.7079303Z inflating: build/lib/torch/utils/model_dump/preact.mjs 2024-08-08T20:48:47.7079647Z creating: build/lib/torch/utils/tensorboard/ 2024-08-08T20:48:47.7080462Z inflating: build/lib/torch/utils/tensorboard/__init__.py 2024-08-08T20:48:47.7081480Z inflating: build/lib/torch/utils/tensorboard/_convert_np.py 2024-08-08T20:48:47.7082583Z inflating: build/lib/torch/utils/tensorboard/_embedding.py 2024-08-08T20:48:47.7083598Z inflating: build/lib/torch/utils/tensorboard/_onnx_graph.py 2024-08-08T20:48:47.7084427Z inflating: build/lib/torch/utils/tensorboard/_proto_graph.py 2024-08-08T20:48:47.7087487Z inflating: build/lib/torch/utils/tensorboard/_pytorch_graph.py 2024-08-08T20:48:47.7088561Z inflating: build/lib/torch/utils/tensorboard/_utils.py 2024-08-08T20:48:47.7092422Z inflating: build/lib/torch/utils/tensorboard/summary.py 2024-08-08T20:48:47.7097108Z inflating: build/lib/torch/utils/tensorboard/writer.py 2024-08-08T20:48:47.7097364Z creating: build/lib/torch/utils/viz/ 2024-08-08T20:48:47.7097795Z extracting: build/lib/torch/utils/viz/__init__.py 2024-08-08T20:48:47.7100162Z inflating: build/lib/torch/utils/viz/_cycles.py 2024-08-08T20:48:47.7100793Z inflating: build/lib/torch/utils/_config_typing.pyi 2024-08-08T20:48:47.7101159Z creating: build/lib/torch/xpu/ 2024-08-08T20:48:47.7103432Z inflating: build/lib/torch/xpu/__init__.py 2024-08-08T20:48:47.7104022Z inflating: build/lib/torch/xpu/_gpu_trace.py 2024-08-08T20:48:47.7105027Z inflating: build/lib/torch/xpu/_utils.py 2024-08-08T20:48:47.7106445Z inflating: build/lib/torch/xpu/random.py 2024-08-08T20:48:47.7107919Z inflating: build/lib/torch/xpu/streams.py 2024-08-08T20:48:47.7108239Z extracting: build/lib/torch/py.typed 2024-08-08T20:48:47.7108589Z creating: build/lib/torch/bin/ 2024-08-08T20:48:47.7653087Z inflating: build/lib/torch/bin/protoc-3.13.0.0 2024-08-08T20:48:47.8196422Z inflating: build/lib/torch/bin/protoc 2024-08-08T20:48:47.8262446Z inflating: build/lib/torch/bin/test_edge_op_registration 2024-08-08T20:48:47.8966513Z inflating: build/lib/torch/bin/test_jit 2024-08-08T20:48:47.9970207Z inflating: build/lib/torch/bin/test_tensorexpr 2024-08-08T20:48:47.9987438Z inflating: build/lib/torch/bin/tutorial_tensorexpr 2024-08-08T20:48:48.0052070Z inflating: build/lib/torch/bin/FileStoreTest 2024-08-08T20:48:48.0120508Z inflating: build/lib/torch/bin/TCPStoreTest 2024-08-08T20:48:48.0186080Z inflating: build/lib/torch/bin/HashStoreTest 2024-08-08T20:48:48.0266413Z inflating: build/lib/torch/bin/ProcessGroupGlooTest 2024-08-08T20:48:48.0344014Z inflating: build/lib/torch/bin/ProcessGroupNCCLTest 2024-08-08T20:48:48.0421246Z inflating: build/lib/torch/bin/ProcessGroupNCCLErrorsTest 2024-08-08T20:48:48.0437838Z inflating: build/lib/torch/bin/ProcessGroupMPITest 2024-08-08T20:48:48.0505356Z inflating: build/lib/torch/bin/test_dist_autograd 2024-08-08T20:48:48.0590187Z inflating: build/lib/torch/bin/test_cpp_rpc 2024-08-08T20:48:48.2022600Z inflating: build/lib/torch/bin/test_api 2024-08-08T20:48:48.2104876Z inflating: build/lib/torch/bin/test_mobile_nnc 2024-08-08T20:48:48.2115722Z inflating: build/lib/torch/bin/aot_model_compiler_test 2024-08-08T20:48:48.2534114Z inflating: build/lib/torch/bin/test_lazy 2024-08-08T20:48:48.2534807Z creating: build/lib/torch/test/ 2024-08-08T20:48:48.2595694Z inflating: build/lib/torch/test/c10_CompileTimeFunctionPointer_test 2024-08-08T20:48:48.2658326Z inflating: build/lib/torch/test/c10_DeviceGuard_test 2024-08-08T20:48:48.2720156Z inflating: build/lib/torch/test/c10_Device_test 2024-08-08T20:48:48.2792109Z inflating: build/lib/torch/test/c10_DispatchKeySet_test 2024-08-08T20:48:48.2856819Z inflating: build/lib/torch/test/c10_Scalar_test 2024-08-08T20:48:48.2916937Z inflating: build/lib/torch/test/c10_StreamGuard_test 2024-08-08T20:48:48.2979899Z inflating: build/lib/torch/test/c10_SymInt_test 2024-08-08T20:48:48.3046275Z inflating: build/lib/torch/test/c10_InlineDeviceGuard_test 2024-08-08T20:48:48.3114485Z inflating: build/lib/torch/test/c10_InlineStreamGuard_test 2024-08-08T20:48:48.3183438Z inflating: build/lib/torch/test/c10_SizesAndStrides_test 2024-08-08T20:48:48.3270351Z inflating: build/lib/torch/test/c10_cow_test 2024-08-08T20:48:48.3334661Z inflating: build/lib/torch/test/c10_Bitset_test 2024-08-08T20:48:48.3394743Z inflating: build/lib/torch/test/c10_ConstexprCrc_test 2024-08-08T20:48:48.3455348Z inflating: build/lib/torch/test/c10_DeadlockDetection_test 2024-08-08T20:48:48.3517533Z inflating: build/lib/torch/test/c10_Half_test 2024-08-08T20:48:48.3586037Z inflating: build/lib/torch/test/c10_LeftRight_test 2024-08-08T20:48:48.3653228Z inflating: build/lib/torch/test/c10_Metaprogramming_test 2024-08-08T20:48:48.3714422Z inflating: build/lib/torch/test/c10_Synchronized_test 2024-08-08T20:48:48.3782656Z inflating: build/lib/torch/test/c10_ThreadLocal_test 2024-08-08T20:48:48.3845342Z inflating: build/lib/torch/test/c10_TypeIndex_test 2024-08-08T20:48:48.3907917Z inflating: build/lib/torch/test/c10_TypeList_test 2024-08-08T20:48:48.3967959Z inflating: build/lib/torch/test/c10_TypeTraits_test 2024-08-08T20:48:48.4031092Z inflating: build/lib/torch/test/c10_accumulate_test 2024-08-08T20:48:48.4099128Z inflating: build/lib/torch/test/c10_bfloat16_test 2024-08-08T20:48:48.4161575Z inflating: build/lib/torch/test/c10_bit_cast_test 2024-08-08T20:48:48.4230441Z inflating: build/lib/torch/test/c10_complex_math_test 2024-08-08T20:48:48.4298468Z inflating: build/lib/torch/test/c10_complex_test 2024-08-08T20:48:48.4363368Z inflating: build/lib/torch/test/c10_exception_test 2024-08-08T20:48:48.4425019Z inflating: build/lib/torch/test/c10_flags_test 2024-08-08T20:48:48.4486202Z inflating: build/lib/torch/test/c10_generic_math_test 2024-08-08T20:48:48.4685219Z inflating: build/lib/torch/test/c10_intrusive_ptr_test 2024-08-08T20:48:48.4747053Z inflating: build/lib/torch/test/c10_irange_test 2024-08-08T20:48:48.4812664Z inflating: build/lib/torch/test/c10_lazy_test 2024-08-08T20:48:48.4882647Z inflating: build/lib/torch/test/c10_logging_test 2024-08-08T20:48:48.4973943Z inflating: build/lib/torch/test/c10_optional_test 2024-08-08T20:48:48.5049811Z inflating: build/lib/torch/test/c10_ordered_preserving_dict_test 2024-08-08T20:48:48.5115604Z inflating: build/lib/torch/test/c10_registry_test 2024-08-08T20:48:48.5298304Z inflating: build/lib/torch/test/c10_small_vector_test 2024-08-08T20:48:48.5360930Z inflating: build/lib/torch/test/c10_ssize_test 2024-08-08T20:48:48.5425189Z inflating: build/lib/torch/test/c10_string_util_test 2024-08-08T20:48:48.5497065Z inflating: build/lib/torch/test/c10_string_view_test 2024-08-08T20:48:48.5558781Z inflating: build/lib/torch/test/c10_tempfile_test 2024-08-08T20:48:48.5627105Z inflating: build/lib/torch/test/c10_typeid_test 2024-08-08T20:48:48.5686701Z inflating: build/lib/torch/test/c10_intrusive_ptr_benchmark 2024-08-08T20:48:48.5751432Z inflating: build/lib/torch/test/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-08T20:48:48.5816015Z inflating: build/lib/torch/test/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-08T20:48:48.5880788Z inflating: build/lib/torch/test/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-08T20:48:48.5943892Z inflating: build/lib/torch/test/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-08T20:48:48.6008847Z inflating: build/lib/torch/test/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-08T20:48:48.6073673Z inflating: build/lib/torch/test/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-08T20:48:48.6137714Z inflating: build/lib/torch/test/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-08T20:48:48.6197647Z inflating: build/lib/torch/test/c10_cuda_CUDATest 2024-08-08T20:48:48.6287780Z inflating: build/lib/torch/test/Dict_test 2024-08-08T20:48:48.6350786Z inflating: build/lib/torch/test/Dimname_test 2024-08-08T20:48:48.6429796Z inflating: build/lib/torch/test/MaybeOwned_test 2024-08-08T20:48:48.6500092Z inflating: build/lib/torch/test/NamedTensor_test 2024-08-08T20:48:48.6572478Z inflating: build/lib/torch/test/apply_utils_test 2024-08-08T20:48:48.6643998Z inflating: build/lib/torch/test/atest 2024-08-08T20:48:48.6721454Z inflating: build/lib/torch/test/basic 2024-08-08T20:48:48.6788694Z inflating: build/lib/torch/test/broadcast_test 2024-08-08T20:48:48.6850554Z inflating: build/lib/torch/test/cpu_allocator_test 2024-08-08T20:48:48.6921790Z inflating: build/lib/torch/test/cpu_generator_test 2024-08-08T20:48:48.6986991Z inflating: build/lib/torch/test/cpu_profiling_allocator_test 2024-08-08T20:48:48.7100493Z inflating: build/lib/torch/test/cpu_rng_test 2024-08-08T20:48:48.7161376Z inflating: build/lib/torch/test/dispatch_key_set_test 2024-08-08T20:48:48.7222930Z inflating: build/lib/torch/test/dlconvertor_test 2024-08-08T20:48:48.7294076Z inflating: build/lib/torch/test/extension_backend_test 2024-08-08T20:48:48.7361710Z inflating: build/lib/torch/test/half_test 2024-08-08T20:48:48.7478499Z inflating: build/lib/torch/test/ivalue_test 2024-08-08T20:48:48.7539392Z inflating: build/lib/torch/test/lazy_tensor_test 2024-08-08T20:48:48.7606049Z inflating: build/lib/torch/test/math_kernel_test 2024-08-08T20:48:48.7672367Z inflating: build/lib/torch/test/memory_format_test 2024-08-08T20:48:48.7736962Z inflating: build/lib/torch/test/memory_overlapping_test 2024-08-08T20:48:48.7802182Z inflating: build/lib/torch/test/mobile_memory_cleanup 2024-08-08T20:48:48.7870655Z inflating: build/lib/torch/test/native_test 2024-08-08T20:48:48.7932783Z inflating: build/lib/torch/test/operator_name_test 2024-08-08T20:48:48.7995689Z inflating: build/lib/torch/test/operators_test 2024-08-08T20:48:48.8059159Z inflating: build/lib/torch/test/packedtensoraccessor_test 2024-08-08T20:48:48.8140841Z inflating: build/lib/torch/test/pow_test 2024-08-08T20:48:48.8211119Z inflating: build/lib/torch/test/quantized_test 2024-08-08T20:48:48.8272405Z inflating: build/lib/torch/test/reduce_ops_test 2024-08-08T20:48:48.8334959Z inflating: build/lib/torch/test/reportMemoryUsage_test 2024-08-08T20:48:48.8404099Z inflating: build/lib/torch/test/scalar_tensor_test 2024-08-08T20:48:48.8475720Z inflating: build/lib/torch/test/scalar_test 2024-08-08T20:48:48.8539141Z inflating: build/lib/torch/test/StorageUtils_test 2024-08-08T20:48:48.8603504Z inflating: build/lib/torch/test/stride_properties_test 2024-08-08T20:48:48.8699312Z inflating: build/lib/torch/test/tensor_iterator_test 2024-08-08T20:48:48.8765702Z inflating: build/lib/torch/test/test_parallel 2024-08-08T20:48:48.8770190Z inflating: build/lib/torch/test/thread_init_test 2024-08-08T20:48:48.8838141Z inflating: build/lib/torch/test/type_ptr_test 2024-08-08T20:48:48.8911643Z inflating: build/lib/torch/test/type_test 2024-08-08T20:48:48.8975694Z inflating: build/lib/torch/test/undefined_tensor_test 2024-08-08T20:48:48.8977258Z inflating: build/lib/torch/test/verify_api_visibility 2024-08-08T20:48:48.9062336Z inflating: build/lib/torch/test/legacy_vmap_test 2024-08-08T20:48:48.9125134Z inflating: build/lib/torch/test/weakref_test 2024-08-08T20:48:48.9188673Z inflating: build/lib/torch/test/wrapdim_test 2024-08-08T20:48:48.9251706Z inflating: build/lib/torch/test/xla_tensor_test 2024-08-08T20:48:48.9325664Z inflating: build/lib/torch/test/IListRef_test 2024-08-08T20:48:48.9454441Z inflating: build/lib/torch/test/List_test 2024-08-08T20:48:48.9535981Z inflating: build/lib/torch/test/KernelFunction_test 2024-08-08T20:48:48.9684243Z inflating: build/lib/torch/test/kernel_function_legacy_test 2024-08-08T20:48:48.9801798Z inflating: build/lib/torch/test/kernel_function_test 2024-08-08T20:48:48.9957354Z inflating: build/lib/torch/test/kernel_lambda_legacy_test 2024-08-08T20:48:49.0083305Z inflating: build/lib/torch/test/kernel_lambda_test 2024-08-08T20:48:49.0157853Z inflating: build/lib/torch/test/kernel_stackbased_test 2024-08-08T20:48:49.0275311Z inflating: build/lib/torch/test/make_boxed_from_unboxed_functor_test 2024-08-08T20:48:49.0337352Z inflating: build/lib/torch/test/CppSignature_test 2024-08-08T20:48:49.0406051Z inflating: build/lib/torch/test/backend_fallback_test 2024-08-08T20:48:49.0466018Z inflating: build/lib/torch/test/op_allowlist_test 2024-08-08T20:48:49.0831868Z inflating: build/lib/torch/test/op_registration_test 2024-08-08T20:48:49.0910001Z inflating: build/lib/torch/test/inline_container_test 2024-08-08T20:48:49.0973614Z inflating: build/lib/torch/test/cuda_allocator_test 2024-08-08T20:48:49.1038122Z inflating: build/lib/torch/test/cuda_apply_test 2024-08-08T20:48:49.1110427Z inflating: build/lib/torch/test/cuda_atomic_ops_test 2024-08-08T20:48:49.1177593Z inflating: build/lib/torch/test/cuda_caching_host_allocator_test 2024-08-08T20:48:49.1262711Z inflating: build/lib/torch/test/cuda_complex_math_test 2024-08-08T20:48:49.1334609Z inflating: build/lib/torch/test/cuda_complex_test 2024-08-08T20:48:49.1406402Z inflating: build/lib/torch/test/cuda_cub_test 2024-08-08T20:48:49.1467091Z inflating: build/lib/torch/test/cuda_device_test 2024-08-08T20:48:49.1547236Z inflating: build/lib/torch/test/cuda_distributions_test 2024-08-08T20:48:49.1609729Z inflating: build/lib/torch/test/cuda_dlconvertor_test 2024-08-08T20:48:49.1679100Z inflating: build/lib/torch/test/cuda_generator_test 2024-08-08T20:48:49.1739773Z inflating: build/lib/torch/test/cuda_half_test 2024-08-08T20:48:49.1802603Z inflating: build/lib/torch/test/cuda_integer_divider_test 2024-08-08T20:48:49.1863357Z inflating: build/lib/torch/test/cuda_optional_test 2024-08-08T20:48:49.1926521Z inflating: build/lib/torch/test/cuda_packedtensoraccessor_test 2024-08-08T20:48:49.1990965Z inflating: build/lib/torch/test/cuda_reportMemoryUsage_test 2024-08-08T20:48:49.2051987Z inflating: build/lib/torch/test/cuda_allocatorTraceTracker_test 2024-08-08T20:48:49.2125888Z inflating: build/lib/torch/test/cuda_stream_test 2024-08-08T20:48:49.2189965Z inflating: build/lib/torch/test/cuda_vectorized_test 2024-08-08T20:48:49.2250077Z inflating: build/lib/torch/test/cuda_cudnn_test 2024-08-08T20:48:49.2251468Z creating: build/lib/torch/_C/ 2024-08-08T20:48:49.2252534Z inflating: build/lib/torch/_C/_aoti.pyi 2024-08-08T20:48:49.2253918Z inflating: build/lib/torch/_C/_autograd.pyi 2024-08-08T20:48:49.2255213Z inflating: build/lib/torch/_C/_cpu.pyi 2024-08-08T20:48:49.2255815Z inflating: build/lib/torch/_C/_cudnn.pyi 2024-08-08T20:48:49.2256511Z inflating: build/lib/torch/_C/_distributed_autograd.pyi 2024-08-08T20:48:49.2258140Z inflating: build/lib/torch/_C/_distributed_c10d.pyi 2024-08-08T20:48:49.2259992Z inflating: build/lib/torch/_C/_distributed_rpc.pyi 2024-08-08T20:48:49.2261684Z inflating: build/lib/torch/_C/_distributed_rpc_testing.pyi 2024-08-08T20:48:49.2263639Z inflating: build/lib/torch/_C/_functions.pyi 2024-08-08T20:48:49.2265125Z inflating: build/lib/torch/_C/_functorch.pyi 2024-08-08T20:48:49.2266479Z inflating: build/lib/torch/_C/_itt.pyi 2024-08-08T20:48:49.2267587Z inflating: build/lib/torch/_C/_lazy.pyi 2024-08-08T20:48:49.2269005Z inflating: build/lib/torch/_C/_lazy_ts_backend.pyi 2024-08-08T20:48:49.2270183Z inflating: build/lib/torch/_C/_monitor.pyi 2024-08-08T20:48:49.2270770Z inflating: build/lib/torch/_C/_nvtx.pyi 2024-08-08T20:48:49.2271355Z inflating: build/lib/torch/_C/_onnx.pyi 2024-08-08T20:48:49.2271965Z inflating: build/lib/torch/_C/_profiler.pyi 2024-08-08T20:48:49.2272572Z inflating: build/lib/torch/_C/_verbose.pyi 2024-08-08T20:48:49.2273160Z creating: build/lib/torch/include/ 2024-08-08T20:48:49.2275881Z inflating: build/lib/torch/include/pthreadpool.h 2024-08-08T20:48:49.2277226Z inflating: build/lib/torch/include/fxdiv.h 2024-08-08T20:48:49.2281997Z inflating: build/lib/torch/include/cpuinfo.h 2024-08-08T20:48:49.2283478Z inflating: build/lib/torch/include/qnnpack_func.h 2024-08-08T20:48:49.2284703Z inflating: build/lib/torch/include/clog.h 2024-08-08T20:48:49.2287754Z inflating: build/lib/torch/include/psimd.h 2024-08-08T20:48:49.2288844Z inflating: build/lib/torch/include/fp16.h 2024-08-08T20:48:49.2291402Z inflating: build/lib/torch/include/nnpack.h 2024-08-08T20:48:49.2305330Z inflating: build/lib/torch/include/xnnpack.h 2024-08-08T20:48:49.2307410Z inflating: build/lib/torch/include/experiments-config.h 2024-08-08T20:48:49.2309021Z inflating: build/lib/torch/include/dnnl.h 2024-08-08T20:48:49.2310454Z inflating: build/lib/torch/include/dnnl_config.h 2024-08-08T20:48:49.2311942Z inflating: build/lib/torch/include/dnnl_debug.h 2024-08-08T20:48:49.2313489Z inflating: build/lib/torch/include/dnnl_ocl.h 2024-08-08T20:48:49.2314864Z inflating: build/lib/torch/include/dnnl_sycl.h 2024-08-08T20:48:49.2315857Z inflating: build/lib/torch/include/dnnl_sycl_types.h 2024-08-08T20:48:49.2316602Z inflating: build/lib/torch/include/dnnl_threadpool.h 2024-08-08T20:48:49.2317302Z inflating: build/lib/torch/include/dnnl_types.h 2024-08-08T20:48:49.2317977Z inflating: build/lib/torch/include/dnnl_version.h 2024-08-08T20:48:49.2329446Z inflating: build/lib/torch/include/sleef.h 2024-08-08T20:48:49.2330819Z creating: build/lib/torch/include/ATen/ 2024-08-08T20:48:49.2331877Z inflating: build/lib/torch/include/ATen/ATen.h 2024-08-08T20:48:49.2333517Z inflating: build/lib/torch/include/ATen/AccumulateType.h 2024-08-08T20:48:49.2335204Z extracting: build/lib/torch/include/ATen/ArrayRef.h 2024-08-08T20:48:49.2336080Z extracting: build/lib/torch/include/ATen/Backend.h 2024-08-08T20:48:49.2337703Z extracting: build/lib/torch/include/ATen/Backtrace.h 2024-08-08T20:48:49.2339662Z inflating: build/lib/torch/include/ATen/BlasBackend.h 2024-08-08T20:48:49.2341377Z inflating: build/lib/torch/include/ATen/CPUApplyUtils.h 2024-08-08T20:48:49.2342443Z inflating: build/lib/torch/include/ATen/CPUFixedAllocator.h 2024-08-08T20:48:49.2343406Z inflating: build/lib/torch/include/ATen/CPUGeneratorImpl.h 2024-08-08T20:48:49.2345288Z inflating: build/lib/torch/include/ATen/CachedTensorUtils.h 2024-08-08T20:48:49.2347111Z inflating: build/lib/torch/include/ATen/CollapseDims.h 2024-08-08T20:48:49.2348698Z inflating: build/lib/torch/include/ATen/Config.h 2024-08-08T20:48:49.2349717Z inflating: build/lib/torch/include/ATen/Context.h 2024-08-08T20:48:49.2350443Z inflating: build/lib/torch/include/ATen/DLConvertor.h 2024-08-08T20:48:49.2351150Z extracting: build/lib/torch/include/ATen/Device.h 2024-08-08T20:48:49.2352424Z inflating: build/lib/torch/include/ATen/DeviceAccelerator.h 2024-08-08T20:48:49.2353598Z inflating: build/lib/torch/include/ATen/DeviceGuard.h 2024-08-08T20:48:49.2354337Z extracting: build/lib/torch/include/ATen/DimVector.h 2024-08-08T20:48:49.2356098Z extracting: build/lib/torch/include/ATen/Dimname.h 2024-08-08T20:48:49.2357644Z inflating: build/lib/torch/include/ATen/Dispatch.h 2024-08-08T20:48:49.2359192Z inflating: build/lib/torch/include/ATen/Dispatch_v2.h 2024-08-08T20:48:49.2360533Z inflating: build/lib/torch/include/ATen/DynamicLibrary.h 2024-08-08T20:48:49.2362296Z inflating: build/lib/torch/include/ATen/EmptyTensor.h 2024-08-08T20:48:49.2363462Z inflating: build/lib/torch/include/ATen/ExpandBase.h 2024-08-08T20:48:49.2364633Z inflating: build/lib/torch/include/ATen/ExpandUtils.h 2024-08-08T20:48:49.2365475Z extracting: build/lib/torch/include/ATen/Formatting.h 2024-08-08T20:48:49.2367149Z inflating: build/lib/torch/include/ATen/FuncTorchTLS.h 2024-08-08T20:48:49.2368950Z inflating: build/lib/torch/include/ATen/FunctionalStorageImpl.h 2024-08-08T20:48:49.2371130Z inflating: build/lib/torch/include/ATen/FunctionalTensorWrapper.h 2024-08-08T20:48:49.2372767Z extracting: build/lib/torch/include/ATen/Generator.h 2024-08-08T20:48:49.2373821Z inflating: build/lib/torch/include/ATen/InferSize.h 2024-08-08T20:48:49.2375649Z inflating: build/lib/torch/include/ATen/InitialTensorOptions.h 2024-08-08T20:48:49.2377428Z extracting: build/lib/torch/include/ATen/Layout.h 2024-08-08T20:48:49.2379235Z inflating: build/lib/torch/include/ATen/LegacyBatchedFallback.h 2024-08-08T20:48:49.2380979Z inflating: build/lib/torch/include/ATen/LegacyBatchedTensorImpl.h 2024-08-08T20:48:49.2382190Z inflating: build/lib/torch/include/ATen/LegacyVmapMode.h 2024-08-08T20:48:49.2383737Z inflating: build/lib/torch/include/ATen/LegacyVmapTransforms.h 2024-08-08T20:48:49.2385637Z inflating: build/lib/torch/include/ATen/LinalgBackend.h 2024-08-08T20:48:49.2387174Z inflating: build/lib/torch/include/ATen/MapAllocator.h 2024-08-08T20:48:49.2388871Z inflating: build/lib/torch/include/ATen/MatrixRef.h 2024-08-08T20:48:49.2390420Z inflating: build/lib/torch/include/ATen/MemoryOverlap.h 2024-08-08T20:48:49.2392178Z extracting: build/lib/torch/include/ATen/NamedTensor.h 2024-08-08T20:48:49.2393970Z inflating: build/lib/torch/include/ATen/NamedTensorUtils.h 2024-08-08T20:48:49.2395779Z inflating: build/lib/torch/include/ATen/NestedTensorImpl.h 2024-08-08T20:48:49.2397585Z inflating: build/lib/torch/include/ATen/NumericUtils.h 2024-08-08T20:48:49.2399286Z inflating: build/lib/torch/include/ATen/OpMathType.h 2024-08-08T20:48:49.2400469Z inflating: build/lib/torch/include/ATen/OpaqueTensorImpl.h 2024-08-08T20:48:49.2402049Z inflating: build/lib/torch/include/ATen/PTThreadPool.h 2024-08-08T20:48:49.2403029Z inflating: build/lib/torch/include/ATen/PadNd.h 2024-08-08T20:48:49.2404420Z inflating: build/lib/torch/include/ATen/Parallel-inl.h 2024-08-08T20:48:49.2405667Z inflating: build/lib/torch/include/ATen/Parallel.h 2024-08-08T20:48:49.2407612Z inflating: build/lib/torch/include/ATen/ParallelFuture.h 2024-08-08T20:48:49.2409284Z inflating: build/lib/torch/include/ATen/ParallelNative.h 2024-08-08T20:48:49.2410795Z inflating: build/lib/torch/include/ATen/ParallelOpenMP.h 2024-08-08T20:48:49.2411678Z inflating: build/lib/torch/include/ATen/PythonTorchFunctionTLS.h 2024-08-08T20:48:49.2413241Z inflating: build/lib/torch/include/ATen/SavedTensorHooks.h 2024-08-08T20:48:49.2414118Z extracting: build/lib/torch/include/ATen/Scalar.h 2024-08-08T20:48:49.2414838Z inflating: build/lib/torch/include/ATen/ScalarOps.h 2024-08-08T20:48:49.2415578Z inflating: build/lib/torch/include/ATen/ScalarType.h 2024-08-08T20:48:49.2416422Z inflating: build/lib/torch/include/ATen/SequenceNumber.h 2024-08-08T20:48:49.2418118Z extracting: build/lib/torch/include/ATen/SmallVector.h 2024-08-08T20:48:49.2419548Z inflating: build/lib/torch/include/ATen/SparseCsrTensorImpl.h 2024-08-08T20:48:49.2421587Z inflating: build/lib/torch/include/ATen/SparseCsrTensorUtils.h 2024-08-08T20:48:49.2423531Z inflating: build/lib/torch/include/ATen/SparseTensorImpl.h 2024-08-08T20:48:49.2425438Z extracting: build/lib/torch/include/ATen/Storage.h 2024-08-08T20:48:49.2426989Z inflating: build/lib/torch/include/ATen/StorageUtils.h 2024-08-08T20:48:49.2428532Z extracting: build/lib/torch/include/ATen/Tensor.h 2024-08-08T20:48:49.2430160Z inflating: build/lib/torch/include/ATen/TensorAccessor.h 2024-08-08T20:48:49.2431748Z inflating: build/lib/torch/include/ATen/TensorGeometry.h 2024-08-08T20:48:49.2433513Z inflating: build/lib/torch/include/ATen/TensorIndexing.h 2024-08-08T20:48:49.2434367Z inflating: build/lib/torch/include/ATen/TensorIterator.h 2024-08-08T20:48:49.2435240Z inflating: build/lib/torch/include/ATen/TensorIteratorInternal.h 2024-08-08T20:48:49.2436074Z inflating: build/lib/torch/include/ATen/TensorMeta.h 2024-08-08T20:48:49.2437691Z inflating: build/lib/torch/include/ATen/TensorNames.h 2024-08-08T20:48:49.2438876Z inflating: build/lib/torch/include/ATen/TensorOperators.h 2024-08-08T20:48:49.2440693Z extracting: build/lib/torch/include/ATen/TensorOptions.h 2024-08-08T20:48:49.2442209Z inflating: build/lib/torch/include/ATen/TensorSubclassLikeUtils.h 2024-08-08T20:48:49.2444103Z inflating: build/lib/torch/include/ATen/TensorUtils.h 2024-08-08T20:48:49.2445698Z inflating: build/lib/torch/include/ATen/ThreadLocalPythonObjects.h 2024-08-08T20:48:49.2446605Z inflating: build/lib/torch/include/ATen/ThreadLocalState.h 2024-08-08T20:48:49.2447989Z inflating: build/lib/torch/include/ATen/TracerMode.h 2024-08-08T20:48:49.2449372Z inflating: build/lib/torch/include/ATen/TypeDefault.h 2024-08-08T20:48:49.2450065Z inflating: build/lib/torch/include/ATen/Utils.h 2024-08-08T20:48:49.2451104Z inflating: build/lib/torch/include/ATen/Version.h 2024-08-08T20:48:49.2452724Z inflating: build/lib/torch/include/ATen/WrapDimUtils.h 2024-08-08T20:48:49.2453553Z inflating: build/lib/torch/include/ATen/WrapDimUtilsMulti.h 2024-08-08T20:48:49.2454398Z inflating: build/lib/torch/include/ATen/autocast_mode.h 2024-08-08T20:48:49.2455689Z inflating: build/lib/torch/include/ATen/ceil_div.h 2024-08-08T20:48:49.2457080Z inflating: build/lib/torch/include/ATen/code_template.h 2024-08-08T20:48:49.2457946Z inflating: build/lib/torch/include/ATen/cpp_custom_type_hack.h 2024-08-08T20:48:49.2459973Z inflating: build/lib/torch/include/ATen/div_rtn.h 2024-08-08T20:48:49.2460962Z inflating: build/lib/torch/include/ATen/dlpack.h 2024-08-08T20:48:49.2461664Z inflating: build/lib/torch/include/ATen/jit_macros.h 2024-08-08T20:48:49.2462839Z inflating: build/lib/torch/include/ATen/jiterator_macros.h 2024-08-08T20:48:49.2464674Z inflating: build/lib/torch/include/ATen/record_function.h 2024-08-08T20:48:49.2465775Z inflating: build/lib/torch/include/ATen/CPUFunctions.h 2024-08-08T20:48:49.2466574Z inflating: build/lib/torch/include/ATen/CPUFunctions_inl.h 2024-08-08T20:48:49.2467790Z inflating: build/lib/torch/include/ATen/CompositeExplicitAutogradFunctions.h 2024-08-08T20:48:49.2469528Z inflating: build/lib/torch/include/ATen/CompositeExplicitAutogradFunctions_inl.h 2024-08-08T20:48:49.2471602Z inflating: build/lib/torch/include/ATen/CompositeExplicitAutogradNonFunctionalFunctions.h 2024-08-08T20:48:49.2473984Z inflating: build/lib/torch/include/ATen/CompositeExplicitAutogradNonFunctionalFunctions_inl.h 2024-08-08T20:48:49.2476093Z inflating: build/lib/torch/include/ATen/CompositeImplicitAutogradFunctions.h 2024-08-08T20:48:49.2477313Z inflating: build/lib/torch/include/ATen/CompositeImplicitAutogradFunctions_inl.h 2024-08-08T20:48:49.2478996Z inflating: build/lib/torch/include/ATen/CompositeImplicitAutogradNestedTensorFunctions.h 2024-08-08T20:48:49.2480944Z inflating: build/lib/torch/include/ATen/CompositeImplicitAutogradNestedTensorFunctions_inl.h 2024-08-08T20:48:49.2482071Z inflating: build/lib/torch/include/ATen/Functions.h 2024-08-08T20:48:49.2482892Z inflating: build/lib/torch/include/ATen/MetaFunctions.h 2024-08-08T20:48:49.2484501Z inflating: build/lib/torch/include/ATen/MetaFunctions_inl.h 2024-08-08T20:48:49.2485480Z inflating: build/lib/torch/include/ATen/MethodOperators.h 2024-08-08T20:48:49.2486321Z inflating: build/lib/torch/include/ATen/NativeFunctions.h 2024-08-08T20:48:49.2487178Z inflating: build/lib/torch/include/ATen/NativeMetaFunctions.h 2024-08-08T20:48:49.2488795Z inflating: build/lib/torch/include/ATen/Operators.h 2024-08-08T20:48:49.2603708Z inflating: build/lib/torch/include/ATen/RedispatchFunctions.h 2024-08-08T20:48:49.2653231Z inflating: build/lib/torch/include/ATen/RegistrationDeclarations.h 2024-08-08T20:48:49.2740235Z inflating: build/lib/torch/include/ATen/VmapGeneratedPlumbing.h 2024-08-08T20:48:49.2741696Z inflating: build/lib/torch/include/ATen/CUDAFunctions.h 2024-08-08T20:48:49.2743533Z inflating: build/lib/torch/include/ATen/CUDAFunctions_inl.h 2024-08-08T20:48:49.2745236Z creating: build/lib/torch/include/ATen/cpu/ 2024-08-08T20:48:49.2746646Z inflating: build/lib/torch/include/ATen/cpu/FlushDenormal.h 2024-08-08T20:48:49.2748410Z inflating: build/lib/torch/include/ATen/cpu/Utils.h 2024-08-08T20:48:49.2749465Z inflating: build/lib/torch/include/ATen/cpu/vml.h 2024-08-08T20:48:49.2750936Z creating: build/lib/torch/include/ATen/cpu/vec/ 2024-08-08T20:48:49.2751660Z creating: build/lib/torch/include/ATen/cpu/vec/vec256/ 2024-08-08T20:48:49.2753432Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/missing_vld1_neon.h 2024-08-08T20:48:49.2754541Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/missing_vst1_neon.h 2024-08-08T20:48:49.2755542Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256.h 2024-08-08T20:48:49.2757433Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_bfloat16.h 2024-08-08T20:48:49.2759656Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_complex_double.h 2024-08-08T20:48:49.2761739Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_complex_float.h 2024-08-08T20:48:49.2763662Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_convert.h 2024-08-08T20:48:49.2765498Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_double.h 2024-08-08T20:48:49.2768018Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_float.h 2024-08-08T20:48:49.2771447Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_float_neon.h 2024-08-08T20:48:49.2773925Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_half_neon.h 2024-08-08T20:48:49.2779136Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_int.h 2024-08-08T20:48:49.2780595Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_mask.h 2024-08-08T20:48:49.2784380Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vec256_qint.h 2024-08-08T20:48:49.2786033Z creating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/ 2024-08-08T20:48:49.2788015Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_bfloat16_vsx.h 2024-08-08T20:48:49.2790030Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_common_vsx.h 2024-08-08T20:48:49.2791472Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_complex_double_vsx.h 2024-08-08T20:48:49.2793933Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_complex_float_vsx.h 2024-08-08T20:48:49.2796451Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_double_vsx.h 2024-08-08T20:48:49.2797965Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_float_vsx.h 2024-08-08T20:48:49.2800222Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_int16_vsx.h 2024-08-08T20:48:49.2802271Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_int32_vsx.h 2024-08-08T20:48:49.2804188Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_int64_vsx.h 2024-08-08T20:48:49.2806161Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_qint32_vsx.h 2024-08-08T20:48:49.2808225Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_qint8_vsx.h 2024-08-08T20:48:49.2809414Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vec256_quint8_vsx.h 2024-08-08T20:48:49.2811471Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/vsx/vsx_helpers.h 2024-08-08T20:48:49.2812422Z creating: build/lib/torch/include/ATen/cpu/vec/vec256/zarch/ 2024-08-08T20:48:49.2819497Z inflating: build/lib/torch/include/ATen/cpu/vec/vec256/zarch/vec256_zarch.h 2024-08-08T20:48:49.2821020Z creating: build/lib/torch/include/ATen/cpu/vec/vec512/ 2024-08-08T20:48:49.2822092Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512.h 2024-08-08T20:48:49.2826977Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512_bfloat16.h 2024-08-08T20:48:49.2829439Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512_complex_double.h 2024-08-08T20:48:49.2833687Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512_complex_float.h 2024-08-08T20:48:49.2835276Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512_convert.h 2024-08-08T20:48:49.2837166Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512_double.h 2024-08-08T20:48:49.2839567Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512_float.h 2024-08-08T20:48:49.2844718Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512_int.h 2024-08-08T20:48:49.2846312Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512_mask.h 2024-08-08T20:48:49.2850339Z inflating: build/lib/torch/include/ATen/cpu/vec/vec512/vec512_qint.h 2024-08-08T20:48:49.2851828Z inflating: build/lib/torch/include/ATen/cpu/vec/functional.h 2024-08-08T20:48:49.2853572Z inflating: build/lib/torch/include/ATen/cpu/vec/functional_base.h 2024-08-08T20:48:49.2855332Z inflating: build/lib/torch/include/ATen/cpu/vec/functional_bfloat16.h 2024-08-08T20:48:49.2857485Z inflating: build/lib/torch/include/ATen/cpu/vec/intrinsics.h 2024-08-08T20:48:49.2858321Z inflating: build/lib/torch/include/ATen/cpu/vec/vec.h 2024-08-08T20:48:49.2861938Z inflating: build/lib/torch/include/ATen/cpu/vec/vec_base.h 2024-08-08T20:48:49.2863787Z inflating: build/lib/torch/include/ATen/cpu/vec/vec_convert.h 2024-08-08T20:48:49.2865505Z inflating: build/lib/torch/include/ATen/cpu/vec/vec_half.h 2024-08-08T20:48:49.2867297Z inflating: build/lib/torch/include/ATen/cpu/vec/vec_mask.h 2024-08-08T20:48:49.2869031Z inflating: build/lib/torch/include/ATen/cpu/vec/vec_n.h 2024-08-08T20:48:49.2870540Z creating: build/lib/torch/include/ATen/core/ 2024-08-08T20:48:49.2872132Z inflating: build/lib/torch/include/ATen/core/ATenGeneral.h 2024-08-08T20:48:49.2873561Z inflating: build/lib/torch/include/ATen/core/ATenOpList.h 2024-08-08T20:48:49.2874535Z inflating: build/lib/torch/include/ATen/core/ATen_fwd.h 2024-08-08T20:48:49.2876457Z inflating: build/lib/torch/include/ATen/core/ATen_pch.h 2024-08-08T20:48:49.2878014Z inflating: build/lib/torch/include/ATen/core/Array.h 2024-08-08T20:48:49.2879088Z inflating: build/lib/torch/include/ATen/core/Backtrace.h 2024-08-08T20:48:49.2880522Z inflating: build/lib/torch/include/ATen/core/CachingHostAllocator.h 2024-08-08T20:48:49.2882643Z inflating: build/lib/torch/include/ATen/core/CheckMemoryFormat.h 2024-08-08T20:48:49.2884465Z inflating: build/lib/torch/include/ATen/core/DeprecatedTypeProperties.h 2024-08-08T20:48:49.2886670Z inflating: build/lib/torch/include/ATen/core/DeprecatedTypePropertiesRegistry.h 2024-08-08T20:48:49.2887808Z inflating: build/lib/torch/include/ATen/core/Dict.h 2024-08-08T20:48:49.2889347Z inflating: build/lib/torch/include/ATen/core/Dict_inl.h 2024-08-08T20:48:49.2890541Z inflating: build/lib/torch/include/ATen/core/DimVector.h 2024-08-08T20:48:49.2891322Z inflating: build/lib/torch/include/ATen/core/Dimname.h 2024-08-08T20:48:49.2893108Z inflating: build/lib/torch/include/ATen/core/DistributionsHelper.h 2024-08-08T20:48:49.2894503Z inflating: build/lib/torch/include/ATen/core/Formatting.h 2024-08-08T20:48:49.2895521Z inflating: build/lib/torch/include/ATen/core/Generator.h 2024-08-08T20:48:49.2896480Z inflating: build/lib/torch/include/ATen/core/GeneratorForPrivateuseone.h 2024-08-08T20:48:49.2897658Z inflating: build/lib/torch/include/ATen/core/IListRef.h 2024-08-08T20:48:49.2898658Z inflating: build/lib/torch/include/ATen/core/IListRef_inl.h 2024-08-08T20:48:49.2899694Z inflating: build/lib/torch/include/ATen/core/LegacyTypeDispatch.h 2024-08-08T20:48:49.2900819Z inflating: build/lib/torch/include/ATen/core/List.h 2024-08-08T20:48:49.2901774Z inflating: build/lib/torch/include/ATen/core/List_inl.h 2024-08-08T20:48:49.2902554Z inflating: build/lib/torch/include/ATen/core/MT19937RNGEngine.h 2024-08-08T20:48:49.2903703Z inflating: build/lib/torch/include/ATen/core/NamedTensor.h 2024-08-08T20:48:49.2904938Z inflating: build/lib/torch/include/ATen/core/NestedIntSymNodeImpl.h 2024-08-08T20:48:49.2906057Z inflating: build/lib/torch/include/ATen/core/PhiloxRNGEngine.h 2024-08-08T20:48:49.2907243Z inflating: build/lib/torch/include/ATen/core/PythonFallbackKernel.h 2024-08-08T20:48:49.2908657Z inflating: build/lib/torch/include/ATen/core/PythonOpRegistrationTrampoline.h 2024-08-08T20:48:49.2910039Z inflating: build/lib/torch/include/ATen/core/QuantizerBase.h 2024-08-08T20:48:49.2911030Z inflating: build/lib/torch/include/ATen/core/Range.h 2024-08-08T20:48:49.2911947Z inflating: build/lib/torch/include/ATen/core/Reduction.h 2024-08-08T20:48:49.2912641Z extracting: build/lib/torch/include/ATen/core/Scalar.h 2024-08-08T20:48:49.2913699Z extracting: build/lib/torch/include/ATen/core/ScalarType.h 2024-08-08T20:48:49.2914395Z inflating: build/lib/torch/include/ATen/core/Tensor.h 2024-08-08T20:48:49.2915193Z inflating: build/lib/torch/include/ATen/core/TensorAccessor.h 2024-08-08T20:48:49.2917892Z inflating: build/lib/torch/include/ATen/core/TensorBase.h 2024-08-08T20:48:49.2918905Z inflating: build/lib/torch/include/ATen/core/TorchDispatchUtils.h 2024-08-08T20:48:49.2920439Z inflating: build/lib/torch/include/ATen/core/TransformationHelper.h 2024-08-08T20:48:49.2921657Z extracting: build/lib/torch/include/ATen/core/UndefinedTensorImpl.h 2024-08-08T20:48:49.2922700Z inflating: build/lib/torch/include/ATen/core/UnsafeFromTH.h 2024-08-08T20:48:49.2923857Z inflating: build/lib/torch/include/ATen/core/VariableHooksInterface.h 2024-08-08T20:48:49.2924946Z inflating: build/lib/torch/include/ATen/core/Variadic.h 2024-08-08T20:48:49.2925721Z inflating: build/lib/torch/include/ATen/core/Vitals.h 2024-08-08T20:48:49.2926641Z inflating: build/lib/torch/include/ATen/core/alias_info.h 2024-08-08T20:48:49.2927588Z inflating: build/lib/torch/include/ATen/core/blob.h 2024-08-08T20:48:49.2928883Z inflating: build/lib/torch/include/ATen/core/builtin_function.h 2024-08-08T20:48:49.2930950Z inflating: build/lib/torch/include/ATen/core/class_type.h 2024-08-08T20:48:49.2931901Z inflating: build/lib/torch/include/ATen/core/custom_class.h 2024-08-08T20:48:49.2933379Z inflating: build/lib/torch/include/ATen/core/dynamic_type.h 2024-08-08T20:48:49.2934464Z inflating: build/lib/torch/include/ATen/core/enum_type.h 2024-08-08T20:48:49.2935679Z inflating: build/lib/torch/include/ATen/core/function.h 2024-08-08T20:48:49.2939017Z inflating: build/lib/torch/include/ATen/core/function_schema.h 2024-08-08T20:48:49.2941150Z inflating: build/lib/torch/include/ATen/core/function_schema_inl.h 2024-08-08T20:48:49.2942202Z inflating: build/lib/torch/include/ATen/core/functional.h 2024-08-08T20:48:49.2943243Z inflating: build/lib/torch/include/ATen/core/grad_mode.h 2024-08-08T20:48:49.2944574Z inflating: build/lib/torch/include/ATen/core/interned_strings.h 2024-08-08T20:48:49.2945901Z inflating: build/lib/torch/include/ATen/core/interned_strings_class.h 2024-08-08T20:48:49.2950999Z inflating: build/lib/torch/include/ATen/core/ivalue.h 2024-08-08T20:48:49.2959338Z inflating: build/lib/torch/include/ATen/core/ivalue_inl.h 2024-08-08T20:48:49.2960198Z inflating: build/lib/torch/include/ATen/core/ivalue_to.h 2024-08-08T20:48:49.2966651Z inflating: build/lib/torch/include/ATen/core/jit_type.h 2024-08-08T20:48:49.2969400Z inflating: build/lib/torch/include/ATen/core/jit_type_base.h 2024-08-08T20:48:49.2970638Z inflating: build/lib/torch/include/ATen/core/operator_name.h 2024-08-08T20:48:49.2971895Z inflating: build/lib/torch/include/ATen/core/qualified_name.h 2024-08-08T20:48:49.2972969Z inflating: build/lib/torch/include/ATen/core/rref_interface.h 2024-08-08T20:48:49.2974071Z inflating: build/lib/torch/include/ATen/core/stack.h 2024-08-08T20:48:49.2975378Z inflating: build/lib/torch/include/ATen/core/symbol.h 2024-08-08T20:48:49.2976510Z inflating: build/lib/torch/include/ATen/core/type_factory.h 2024-08-08T20:48:49.2977631Z inflating: build/lib/torch/include/ATen/core/type_ptr.h 2024-08-08T20:48:49.2978608Z extracting: build/lib/torch/include/ATen/core/typeid.h 2024-08-08T20:48:49.2997379Z inflating: build/lib/torch/include/ATen/core/TensorBody.h 2024-08-08T20:48:49.3002638Z inflating: build/lib/torch/include/ATen/core/aten_interned_strings.h 2024-08-08T20:48:49.3003707Z inflating: build/lib/torch/include/ATen/core/enum_tag.h 2024-08-08T20:48:49.3004660Z creating: build/lib/torch/include/ATen/core/boxing/ 2024-08-08T20:48:49.3006028Z inflating: build/lib/torch/include/ATen/core/boxing/BoxedKernel.h 2024-08-08T20:48:49.3007171Z inflating: build/lib/torch/include/ATen/core/boxing/BoxedKernel_impl.h 2024-08-08T20:48:49.3008713Z inflating: build/lib/torch/include/ATen/core/boxing/KernelFunction.h 2024-08-08T20:48:49.3010913Z inflating: build/lib/torch/include/ATen/core/boxing/KernelFunction_impl.h 2024-08-08T20:48:49.3012283Z inflating: build/lib/torch/include/ATen/core/boxing/OperatorKernel.h 2024-08-08T20:48:49.3013419Z creating: build/lib/torch/include/ATen/core/boxing/impl/ 2024-08-08T20:48:49.3014578Z inflating: build/lib/torch/include/ATen/core/boxing/impl/WrapFunctionIntoFunctor.h 2024-08-08T20:48:49.3016397Z inflating: build/lib/torch/include/ATen/core/boxing/impl/WrapFunctionIntoRuntimeFunctor.h 2024-08-08T20:48:49.3017838Z inflating: build/lib/torch/include/ATen/core/boxing/impl/boxing.h 2024-08-08T20:48:49.3019978Z inflating: build/lib/torch/include/ATen/core/boxing/impl/make_boxed_from_unboxed_functor.h 2024-08-08T20:48:49.3022546Z inflating: build/lib/torch/include/ATen/core/boxing/impl/test_helpers.h 2024-08-08T20:48:49.3023919Z creating: build/lib/torch/include/ATen/core/dispatch/ 2024-08-08T20:48:49.3025173Z inflating: build/lib/torch/include/ATen/core/dispatch/CppSignature.h 2024-08-08T20:48:49.3026275Z inflating: build/lib/torch/include/ATen/core/dispatch/DispatchKeyExtractor.h 2024-08-08T20:48:49.3028441Z inflating: build/lib/torch/include/ATen/core/dispatch/Dispatcher.h 2024-08-08T20:48:49.3029658Z inflating: build/lib/torch/include/ATen/core/dispatch/ObservedOperators.h 2024-08-08T20:48:49.3031958Z inflating: build/lib/torch/include/ATen/core/dispatch/OperatorEntry.h 2024-08-08T20:48:49.3034236Z inflating: build/lib/torch/include/ATen/core/dispatch/OperatorOptions.h 2024-08-08T20:48:49.3036470Z inflating: build/lib/torch/include/ATen/core/dispatch/RegistrationHandleRAII.h 2024-08-08T20:48:49.3038201Z creating: build/lib/torch/include/ATen/core/op_registration/ 2024-08-08T20:48:49.3039277Z inflating: build/lib/torch/include/ATen/core/op_registration/adaption.h 2024-08-08T20:48:49.3041657Z inflating: build/lib/torch/include/ATen/core/op_registration/infer_schema.h 2024-08-08T20:48:49.3042987Z inflating: build/lib/torch/include/ATen/core/op_registration/op_allowlist.h 2024-08-08T20:48:49.3044114Z inflating: build/lib/torch/include/ATen/core/op_registration/op_registration.h 2024-08-08T20:48:49.3045290Z creating: build/lib/torch/include/ATen/cuda/ 2024-08-08T20:48:49.3045973Z inflating: build/lib/torch/include/ATen/cuda/ApplyGridUtils.cuh 2024-08-08T20:48:49.3047450Z inflating: build/lib/torch/include/ATen/cuda/AsmUtils.cuh 2024-08-08T20:48:49.3048485Z inflating: build/lib/torch/include/ATen/cuda/Atomic.cuh 2024-08-08T20:48:49.3049428Z inflating: build/lib/torch/include/ATen/cuda/CUDAApplyUtils.cuh 2024-08-08T20:48:49.3050611Z inflating: build/lib/torch/include/ATen/cuda/CUDAGraphsUtils.cuh 2024-08-08T20:48:49.3051731Z inflating: build/lib/torch/include/ATen/cuda/CUDATensorMethods.cuh 2024-08-08T20:48:49.3052819Z inflating: build/lib/torch/include/ATen/cuda/DeviceUtils.cuh 2024-08-08T20:48:49.3053897Z inflating: build/lib/torch/include/ATen/cuda/NumericLimits.cuh 2024-08-08T20:48:49.3054693Z inflating: build/lib/torch/include/ATen/cuda/PhiloxUtils.cuh 2024-08-08T20:48:49.3055790Z inflating: build/lib/torch/include/ATen/cuda/ScanUtils.cuh 2024-08-08T20:48:49.3056760Z inflating: build/lib/torch/include/ATen/cuda/cub.cuh 2024-08-08T20:48:49.3057763Z inflating: build/lib/torch/include/ATen/cuda/cub_definitions.cuh 2024-08-08T20:48:49.3059013Z inflating: build/lib/torch/include/ATen/cuda/ATenCUDAGeneral.h 2024-08-08T20:48:49.3060255Z inflating: build/lib/torch/include/ATen/cuda/CUDABlas.h 2024-08-08T20:48:49.3061082Z inflating: build/lib/torch/include/ATen/cuda/CUDAConfig.h 2024-08-08T20:48:49.3062101Z inflating: build/lib/torch/include/ATen/cuda/CUDAContext.h 2024-08-08T20:48:49.3063256Z inflating: build/lib/torch/include/ATen/cuda/CUDAContextLight.h 2024-08-08T20:48:49.3064305Z inflating: build/lib/torch/include/ATen/cuda/CUDADataType.h 2024-08-08T20:48:49.3065111Z inflating: build/lib/torch/include/ATen/cuda/CUDADevice.h 2024-08-08T20:48:49.3066172Z inflating: build/lib/torch/include/ATen/cuda/CUDAEvent.h 2024-08-08T20:48:49.3067376Z inflating: build/lib/torch/include/ATen/cuda/CUDAGeneratorImpl.h 2024-08-08T20:48:49.3068604Z inflating: build/lib/torch/include/ATen/cuda/CUDAGraph.h 2024-08-08T20:48:49.3069642Z inflating: build/lib/torch/include/ATen/cuda/CUDASparse.h 2024-08-08T20:48:49.3071737Z inflating: build/lib/torch/include/ATen/cuda/CUDASparseBlas.h 2024-08-08T20:48:49.3072976Z inflating: build/lib/torch/include/ATen/cuda/CUDASparseDescriptors.h 2024-08-08T20:48:49.3074104Z inflating: build/lib/torch/include/ATen/cuda/CUDAUtils.h 2024-08-08T20:48:49.3075204Z inflating: build/lib/torch/include/ATen/cuda/CachingHostAllocator.h 2024-08-08T20:48:49.3076167Z inflating: build/lib/torch/include/ATen/cuda/EmptyTensor.h 2024-08-08T20:48:49.3077382Z inflating: build/lib/torch/include/ATen/cuda/Exceptions.h 2024-08-08T20:48:49.3078451Z inflating: build/lib/torch/include/ATen/cuda/PeerToPeerAccess.h 2024-08-08T20:48:49.3079729Z inflating: build/lib/torch/include/ATen/cuda/PhiloxCudaState.h 2024-08-08T20:48:49.3080860Z inflating: build/lib/torch/include/ATen/cuda/PinnedMemoryAllocator.h 2024-08-08T20:48:49.3082017Z inflating: build/lib/torch/include/ATen/cuda/Sleep.h 2024-08-08T20:48:49.3083273Z inflating: build/lib/torch/include/ATen/cuda/ThrustAllocator.h 2024-08-08T20:48:49.3083997Z inflating: build/lib/torch/include/ATen/cuda/cub.h 2024-08-08T20:48:49.3085010Z inflating: build/lib/torch/include/ATen/cuda/jiterator.h 2024-08-08T20:48:49.3086194Z inflating: build/lib/torch/include/ATen/cuda/jiterator_impl.h 2024-08-08T20:48:49.3087005Z inflating: build/lib/torch/include/ATen/cuda/llvm_jit_strings.h 2024-08-08T20:48:49.3088079Z creating: build/lib/torch/include/ATen/cuda/detail/ 2024-08-08T20:48:49.3089026Z inflating: build/lib/torch/include/ATen/cuda/detail/IndexUtils.cuh 2024-08-08T20:48:49.3090224Z inflating: build/lib/torch/include/ATen/cuda/detail/IntegerDivider.cuh 2024-08-08T20:48:49.3091522Z inflating: build/lib/torch/include/ATen/cuda/detail/OffsetCalculator.cuh 2024-08-08T20:48:49.3092840Z inflating: build/lib/torch/include/ATen/cuda/detail/PhiloxCudaStateRaw.cuh 2024-08-08T20:48:49.3094057Z inflating: build/lib/torch/include/ATen/cuda/detail/TensorInfo.cuh 2024-08-08T20:48:49.3095242Z inflating: build/lib/torch/include/ATen/cuda/detail/UnpackRaw.cuh 2024-08-08T20:48:49.3096305Z inflating: build/lib/torch/include/ATen/cuda/detail/CUDAHooks.h 2024-08-08T20:48:49.3097653Z inflating: build/lib/torch/include/ATen/cuda/detail/DeviceThreadHandles.h 2024-08-08T20:48:49.3099005Z inflating: build/lib/torch/include/ATen/cuda/detail/KernelUtils.h 2024-08-08T20:48:49.3099843Z inflating: build/lib/torch/include/ATen/cuda/detail/LazyNVRTC.h 2024-08-08T20:48:49.3100931Z creating: build/lib/torch/include/ATen/cuda/tunable/ 2024-08-08T20:48:49.3101689Z inflating: build/lib/torch/include/ATen/cuda/tunable/GemmCommon.h 2024-08-08T20:48:49.3102918Z inflating: build/lib/torch/include/ATen/cuda/tunable/GemmHipblaslt.h 2024-08-08T20:48:49.3104285Z inflating: build/lib/torch/include/ATen/cuda/tunable/GemmRocblas.h 2024-08-08T20:48:49.3105353Z inflating: build/lib/torch/include/ATen/cuda/tunable/StreamTimer.h 2024-08-08T20:48:49.3106773Z inflating: build/lib/torch/include/ATen/cuda/tunable/Tunable.h 2024-08-08T20:48:49.3107918Z inflating: build/lib/torch/include/ATen/cuda/tunable/TunableGemm.h 2024-08-08T20:48:49.3109621Z inflating: build/lib/torch/include/ATen/cuda/tunable/TunableOp.h 2024-08-08T20:48:49.3110432Z creating: build/lib/torch/include/ATen/cudnn/ 2024-08-08T20:48:49.3112630Z inflating: build/lib/torch/include/ATen/cudnn/Descriptors.h 2024-08-08T20:48:49.3113507Z extracting: build/lib/torch/include/ATen/cudnn/Exceptions.h 2024-08-08T20:48:49.3114318Z inflating: build/lib/torch/include/ATen/cudnn/Handle.h 2024-08-08T20:48:49.3115102Z extracting: build/lib/torch/include/ATen/cudnn/Handles.h 2024-08-08T20:48:49.3115864Z inflating: build/lib/torch/include/ATen/cudnn/Types.h 2024-08-08T20:48:49.3116616Z inflating: build/lib/torch/include/ATen/cudnn/Utils.h 2024-08-08T20:48:49.3117651Z inflating: build/lib/torch/include/ATen/cudnn/cudnn-wrapper.h 2024-08-08T20:48:49.3118435Z creating: build/lib/torch/include/ATen/functorch/ 2024-08-08T20:48:49.3119277Z inflating: build/lib/torch/include/ATen/functorch/ADInterpreters.h 2024-08-08T20:48:49.3120685Z inflating: build/lib/torch/include/ATen/functorch/BatchRulesHelper.h 2024-08-08T20:48:49.3121837Z inflating: build/lib/torch/include/ATen/functorch/BatchedFallback.h 2024-08-08T20:48:49.3123189Z inflating: build/lib/torch/include/ATen/functorch/BatchedTensorImpl.h 2024-08-08T20:48:49.3124474Z inflating: build/lib/torch/include/ATen/functorch/BatchingMetaprogramming.h 2024-08-08T20:48:49.3125796Z inflating: build/lib/torch/include/ATen/functorch/DynamicLayer.h 2024-08-08T20:48:49.3127090Z inflating: build/lib/torch/include/ATen/functorch/FunctionalizeInterpreter.h 2024-08-08T20:48:49.3128598Z inflating: build/lib/torch/include/ATen/functorch/Interpreter.h 2024-08-08T20:48:49.3130669Z inflating: build/lib/torch/include/ATen/functorch/LegacyVmapTransforms.h 2024-08-08T20:48:49.3131657Z inflating: build/lib/torch/include/ATen/functorch/Macros.h 2024-08-08T20:48:49.3132689Z inflating: build/lib/torch/include/ATen/functorch/PlumbingHelper.h 2024-08-08T20:48:49.3133856Z inflating: build/lib/torch/include/ATen/functorch/TensorWrapper.h 2024-08-08T20:48:49.3134969Z inflating: build/lib/torch/include/ATen/functorch/VmapInterpreter.h 2024-08-08T20:48:49.3135796Z creating: build/lib/torch/include/ATen/ops/ 2024-08-08T20:48:49.3136599Z inflating: build/lib/torch/include/ATen/ops/from_blob.h 2024-08-08T20:48:49.3137646Z inflating: build/lib/torch/include/ATen/ops/tensor.h 2024-08-08T20:48:49.3138840Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d.h 2024-08-08T20:48:49.3140026Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_backward.h 2024-08-08T20:48:49.3141733Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3143526Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.3145232Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.3146759Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_backward_native.h 2024-08-08T20:48:49.3148288Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_backward_ops.h 2024-08-08T20:48:49.3149762Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3151410Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_cpu_dispatch.h 2024-08-08T20:48:49.3152903Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_cuda_dispatch.h 2024-08-08T20:48:49.3154201Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_native.h 2024-08-08T20:48:49.3155420Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool2d_ops.h 2024-08-08T20:48:49.3156723Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d.h 2024-08-08T20:48:49.3157986Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_backward.h 2024-08-08T20:48:49.3160092Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3161881Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_backward_cpu_dispatch.h 2024-08-08T20:48:49.3163340Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_backward_cuda_dispatch.h 2024-08-08T20:48:49.3164935Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_backward_native.h 2024-08-08T20:48:49.3166295Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_backward_ops.h 2024-08-08T20:48:49.3167916Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3169584Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_cpu_dispatch.h 2024-08-08T20:48:49.3170929Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_cuda_dispatch.h 2024-08-08T20:48:49.3172224Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_native.h 2024-08-08T20:48:49.3173805Z inflating: build/lib/torch/include/ATen/ops/_adaptive_avg_pool3d_ops.h 2024-08-08T20:48:49.3175913Z inflating: build/lib/torch/include/ATen/ops/_add_batch_dim.h 2024-08-08T20:48:49.3178453Z inflating: build/lib/torch/include/ATen/ops/_add_batch_dim_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3181067Z inflating: build/lib/torch/include/ATen/ops/_add_batch_dim_native.h 2024-08-08T20:48:49.3183192Z inflating: build/lib/torch/include/ATen/ops/_add_batch_dim_ops.h 2024-08-08T20:48:49.3185074Z inflating: build/lib/torch/include/ATen/ops/_add_relu.h 2024-08-08T20:48:49.3187703Z inflating: build/lib/torch/include/ATen/ops/_add_relu_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3190412Z inflating: build/lib/torch/include/ATen/ops/_add_relu_cpu_dispatch.h 2024-08-08T20:48:49.3192614Z inflating: build/lib/torch/include/ATen/ops/_add_relu_meta_dispatch.h 2024-08-08T20:48:49.3194367Z inflating: build/lib/torch/include/ATen/ops/_add_relu_native.h 2024-08-08T20:48:49.3196244Z inflating: build/lib/torch/include/ATen/ops/_add_relu_ops.h 2024-08-08T20:48:49.3198197Z inflating: build/lib/torch/include/ATen/ops/_addmm_activation.h 2024-08-08T20:48:49.3201092Z inflating: build/lib/torch/include/ATen/ops/_addmm_activation_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.3204223Z inflating: build/lib/torch/include/ATen/ops/_addmm_activation_cpu_dispatch.h 2024-08-08T20:48:49.3206730Z inflating: build/lib/torch/include/ATen/ops/_addmm_activation_cuda_dispatch.h 2024-08-08T20:48:49.3209121Z inflating: build/lib/torch/include/ATen/ops/_addmm_activation_meta.h 2024-08-08T20:48:49.3211567Z inflating: build/lib/torch/include/ATen/ops/_addmm_activation_meta_dispatch.h 2024-08-08T20:48:49.3214204Z inflating: build/lib/torch/include/ATen/ops/_addmm_activation_native.h 2024-08-08T20:48:49.3216479Z inflating: build/lib/torch/include/ATen/ops/_addmm_activation_ops.h 2024-08-08T20:48:49.3218435Z inflating: build/lib/torch/include/ATen/ops/_aminmax.h 2024-08-08T20:48:49.3220734Z inflating: build/lib/torch/include/ATen/ops/_aminmax_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3223347Z inflating: build/lib/torch/include/ATen/ops/_aminmax_cpu_dispatch.h 2024-08-08T20:48:49.3225505Z inflating: build/lib/torch/include/ATen/ops/_aminmax_cuda_dispatch.h 2024-08-08T20:48:49.3227619Z inflating: build/lib/torch/include/ATen/ops/_aminmax_native.h 2024-08-08T20:48:49.3229489Z inflating: build/lib/torch/include/ATen/ops/_aminmax_ops.h 2024-08-08T20:48:49.3231805Z inflating: build/lib/torch/include/ATen/ops/_amp_foreach_non_finite_check_and_unscale.h 2024-08-08T20:48:49.3235224Z inflating: build/lib/torch/include/ATen/ops/_amp_foreach_non_finite_check_and_unscale_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3238831Z inflating: build/lib/torch/include/ATen/ops/_amp_foreach_non_finite_check_and_unscale_cpu_dispatch.h 2024-08-08T20:48:49.3242009Z inflating: build/lib/torch/include/ATen/ops/_amp_foreach_non_finite_check_and_unscale_cuda_dispatch.h 2024-08-08T20:48:49.3245111Z inflating: build/lib/torch/include/ATen/ops/_amp_foreach_non_finite_check_and_unscale_native.h 2024-08-08T20:48:49.3248109Z inflating: build/lib/torch/include/ATen/ops/_amp_foreach_non_finite_check_and_unscale_ops.h 2024-08-08T20:48:49.3250572Z inflating: build/lib/torch/include/ATen/ops/_amp_update_scale.h 2024-08-08T20:48:49.3253220Z inflating: build/lib/torch/include/ATen/ops/_amp_update_scale_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3256103Z inflating: build/lib/torch/include/ATen/ops/_amp_update_scale_cpu_dispatch.h 2024-08-08T20:48:49.3258746Z inflating: build/lib/torch/include/ATen/ops/_amp_update_scale_cuda_dispatch.h 2024-08-08T20:48:49.3260712Z inflating: build/lib/torch/include/ATen/ops/_amp_update_scale_meta_dispatch.h 2024-08-08T20:48:49.3262073Z inflating: build/lib/torch/include/ATen/ops/_amp_update_scale_native.h 2024-08-08T20:48:49.3263229Z inflating: build/lib/torch/include/ATen/ops/_amp_update_scale_ops.h 2024-08-08T20:48:49.3264410Z inflating: build/lib/torch/include/ATen/ops/_assert_async.h 2024-08-08T20:48:49.3265569Z inflating: build/lib/torch/include/ATen/ops/_assert_async_cpu_dispatch.h 2024-08-08T20:48:49.3266818Z inflating: build/lib/torch/include/ATen/ops/_assert_async_cuda_dispatch.h 2024-08-08T20:48:49.3267985Z inflating: build/lib/torch/include/ATen/ops/_assert_async_native.h 2024-08-08T20:48:49.3269269Z inflating: build/lib/torch/include/ATen/ops/_assert_async_ops.h 2024-08-08T20:48:49.3270321Z inflating: build/lib/torch/include/ATen/ops/_assert_scalar.h 2024-08-08T20:48:49.3271800Z inflating: build/lib/torch/include/ATen/ops/_assert_scalar_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3273321Z inflating: build/lib/torch/include/ATen/ops/_assert_scalar_native.h 2024-08-08T20:48:49.3274512Z inflating: build/lib/torch/include/ATen/ops/_assert_scalar_ops.h 2024-08-08T20:48:49.3275551Z inflating: build/lib/torch/include/ATen/ops/_assert_tensor_metadata.h 2024-08-08T20:48:49.3277140Z inflating: build/lib/torch/include/ATen/ops/_assert_tensor_metadata_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3278792Z inflating: build/lib/torch/include/ATen/ops/_assert_tensor_metadata_native.h 2024-08-08T20:48:49.3280046Z inflating: build/lib/torch/include/ATen/ops/_assert_tensor_metadata_ops.h 2024-08-08T20:48:49.3281303Z inflating: build/lib/torch/include/ATen/ops/_autocast_to_full_precision.h 2024-08-08T20:48:49.3282984Z inflating: build/lib/torch/include/ATen/ops/_autocast_to_full_precision_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3284649Z inflating: build/lib/torch/include/ATen/ops/_autocast_to_full_precision_native.h 2024-08-08T20:48:49.3286171Z inflating: build/lib/torch/include/ATen/ops/_autocast_to_full_precision_ops.h 2024-08-08T20:48:49.3287539Z inflating: build/lib/torch/include/ATen/ops/_autocast_to_reduced_precision.h 2024-08-08T20:48:49.3289215Z inflating: build/lib/torch/include/ATen/ops/_autocast_to_reduced_precision_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3290887Z inflating: build/lib/torch/include/ATen/ops/_autocast_to_reduced_precision_native.h 2024-08-08T20:48:49.3292308Z inflating: build/lib/torch/include/ATen/ops/_autocast_to_reduced_precision_ops.h 2024-08-08T20:48:49.3293477Z inflating: build/lib/torch/include/ATen/ops/_backward.h 2024-08-08T20:48:49.3294852Z inflating: build/lib/torch/include/ATen/ops/_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3296149Z inflating: build/lib/torch/include/ATen/ops/_backward_native.h 2024-08-08T20:48:49.3297187Z inflating: build/lib/torch/include/ATen/ops/_backward_ops.h 2024-08-08T20:48:49.3298383Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_impl_index.h 2024-08-08T20:48:49.3299610Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_impl_index_backward.h 2024-08-08T20:48:49.3301149Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_impl_index_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3302889Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_impl_index_backward_native.h 2024-08-08T20:48:49.3304432Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_impl_index_backward_ops.h 2024-08-08T20:48:49.3305944Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_impl_index_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3307565Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_impl_index_native.h 2024-08-08T20:48:49.3308811Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_impl_index_ops.h 2024-08-08T20:48:49.3310161Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_no_update.h 2024-08-08T20:48:49.3311591Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_no_update_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3313133Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_no_update_native.h 2024-08-08T20:48:49.3314388Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_no_update_ops.h 2024-08-08T20:48:49.3315559Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_with_update.h 2024-08-08T20:48:49.3317092Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_with_update_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3318614Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_with_update_cpu_dispatch.h 2024-08-08T20:48:49.3320153Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_with_update_cuda_dispatch.h 2024-08-08T20:48:49.3321468Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_with_update_native.h 2024-08-08T20:48:49.3322846Z inflating: build/lib/torch/include/ATen/ops/_batch_norm_with_update_ops.h 2024-08-08T20:48:49.3323967Z inflating: build/lib/torch/include/ATen/ops/_cast_Byte.h 2024-08-08T20:48:49.3325359Z inflating: build/lib/torch/include/ATen/ops/_cast_Byte_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3326667Z inflating: build/lib/torch/include/ATen/ops/_cast_Byte_native.h 2024-08-08T20:48:49.3327718Z inflating: build/lib/torch/include/ATen/ops/_cast_Byte_ops.h 2024-08-08T20:48:49.3328681Z inflating: build/lib/torch/include/ATen/ops/_cast_Char.h 2024-08-08T20:48:49.3329911Z inflating: build/lib/torch/include/ATen/ops/_cast_Char_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3331361Z inflating: build/lib/torch/include/ATen/ops/_cast_Char_native.h 2024-08-08T20:48:49.3332400Z inflating: build/lib/torch/include/ATen/ops/_cast_Char_ops.h 2024-08-08T20:48:49.3333406Z inflating: build/lib/torch/include/ATen/ops/_cast_Double.h 2024-08-08T20:48:49.3334656Z inflating: build/lib/torch/include/ATen/ops/_cast_Double_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3336231Z inflating: build/lib/torch/include/ATen/ops/_cast_Double_native.h 2024-08-08T20:48:49.3337325Z inflating: build/lib/torch/include/ATen/ops/_cast_Double_ops.h 2024-08-08T20:48:49.3338330Z inflating: build/lib/torch/include/ATen/ops/_cast_Float.h 2024-08-08T20:48:49.3339739Z inflating: build/lib/torch/include/ATen/ops/_cast_Float_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3341023Z inflating: build/lib/torch/include/ATen/ops/_cast_Float_native.h 2024-08-08T20:48:49.3342100Z inflating: build/lib/torch/include/ATen/ops/_cast_Float_ops.h 2024-08-08T20:48:49.3343092Z inflating: build/lib/torch/include/ATen/ops/_cast_Half.h 2024-08-08T20:48:49.3344463Z inflating: build/lib/torch/include/ATen/ops/_cast_Half_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3345768Z inflating: build/lib/torch/include/ATen/ops/_cast_Half_native.h 2024-08-08T20:48:49.3346810Z inflating: build/lib/torch/include/ATen/ops/_cast_Half_ops.h 2024-08-08T20:48:49.3347650Z inflating: build/lib/torch/include/ATen/ops/_cast_Int.h 2024-08-08T20:48:49.3348849Z inflating: build/lib/torch/include/ATen/ops/_cast_Int_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3350300Z inflating: build/lib/torch/include/ATen/ops/_cast_Int_native.h 2024-08-08T20:48:49.3351260Z inflating: build/lib/torch/include/ATen/ops/_cast_Int_ops.h 2024-08-08T20:48:49.3352207Z inflating: build/lib/torch/include/ATen/ops/_cast_Long.h 2024-08-08T20:48:49.3353531Z inflating: build/lib/torch/include/ATen/ops/_cast_Long_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3354820Z inflating: build/lib/torch/include/ATen/ops/_cast_Long_native.h 2024-08-08T20:48:49.3355859Z inflating: build/lib/torch/include/ATen/ops/_cast_Long_ops.h 2024-08-08T20:48:49.3356915Z inflating: build/lib/torch/include/ATen/ops/_cast_Short.h 2024-08-08T20:48:49.3358196Z inflating: build/lib/torch/include/ATen/ops/_cast_Short_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3359974Z inflating: build/lib/torch/include/ATen/ops/_cast_Short_native.h 2024-08-08T20:48:49.3361035Z inflating: build/lib/torch/include/ATen/ops/_cast_Short_ops.h 2024-08-08T20:48:49.3362082Z inflating: build/lib/torch/include/ATen/ops/_cdist_backward.h 2024-08-08T20:48:49.3363385Z inflating: build/lib/torch/include/ATen/ops/_cdist_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3364942Z inflating: build/lib/torch/include/ATen/ops/_cdist_backward_cpu_dispatch.h 2024-08-08T20:48:49.3366197Z inflating: build/lib/torch/include/ATen/ops/_cdist_backward_cuda_dispatch.h 2024-08-08T20:48:49.3367485Z inflating: build/lib/torch/include/ATen/ops/_cdist_backward_native.h 2024-08-08T20:48:49.3368668Z inflating: build/lib/torch/include/ATen/ops/_cdist_backward_ops.h 2024-08-08T20:48:49.3369878Z inflating: build/lib/torch/include/ATen/ops/_cdist_forward.h 2024-08-08T20:48:49.3371354Z inflating: build/lib/torch/include/ATen/ops/_cdist_forward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3372763Z inflating: build/lib/torch/include/ATen/ops/_cdist_forward_cpu_dispatch.h 2024-08-08T20:48:49.3374159Z inflating: build/lib/torch/include/ATen/ops/_cdist_forward_cuda_dispatch.h 2024-08-08T20:48:49.3375359Z inflating: build/lib/torch/include/ATen/ops/_cdist_forward_native.h 2024-08-08T20:48:49.3376461Z inflating: build/lib/torch/include/ATen/ops/_cdist_forward_ops.h 2024-08-08T20:48:49.3377561Z inflating: build/lib/torch/include/ATen/ops/_cholesky_solve_helper.h 2024-08-08T20:48:49.3379204Z inflating: build/lib/torch/include/ATen/ops/_cholesky_solve_helper_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3380715Z inflating: build/lib/torch/include/ATen/ops/_cholesky_solve_helper_cpu_dispatch.h 2024-08-08T20:48:49.3382079Z inflating: build/lib/torch/include/ATen/ops/_cholesky_solve_helper_cuda_dispatch.h 2024-08-08T20:48:49.3383410Z inflating: build/lib/torch/include/ATen/ops/_cholesky_solve_helper_native.h 2024-08-08T20:48:49.3384901Z inflating: build/lib/torch/include/ATen/ops/_cholesky_solve_helper_ops.h 2024-08-08T20:48:49.3386151Z inflating: build/lib/torch/include/ATen/ops/_choose_qparams_per_tensor.h 2024-08-08T20:48:49.3387641Z inflating: build/lib/torch/include/ATen/ops/_choose_qparams_per_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3389309Z inflating: build/lib/torch/include/ATen/ops/_choose_qparams_per_tensor_native.h 2024-08-08T20:48:49.3390653Z inflating: build/lib/torch/include/ATen/ops/_choose_qparams_per_tensor_ops.h 2024-08-08T20:48:49.3391942Z inflating: build/lib/torch/include/ATen/ops/_chunk_cat.h 2024-08-08T20:48:49.3393153Z inflating: build/lib/torch/include/ATen/ops/_chunk_cat_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3394488Z inflating: build/lib/torch/include/ATen/ops/_chunk_cat_cuda_dispatch.h 2024-08-08T20:48:49.3395645Z inflating: build/lib/torch/include/ATen/ops/_chunk_cat_native.h 2024-08-08T20:48:49.3396690Z inflating: build/lib/torch/include/ATen/ops/_chunk_cat_ops.h 2024-08-08T20:48:49.3397691Z inflating: build/lib/torch/include/ATen/ops/_coalesce.h 2024-08-08T20:48:49.3398912Z inflating: build/lib/torch/include/ATen/ops/_coalesce_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3400342Z inflating: build/lib/torch/include/ATen/ops/_coalesce_native.h 2024-08-08T20:48:49.3401369Z inflating: build/lib/torch/include/ATen/ops/_coalesce_ops.h 2024-08-08T20:48:49.3402368Z inflating: build/lib/torch/include/ATen/ops/_coalesced.h 2024-08-08T20:48:49.3403594Z inflating: build/lib/torch/include/ATen/ops/_coalesced_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3404930Z inflating: build/lib/torch/include/ATen/ops/_coalesced_meta_dispatch.h 2024-08-08T20:48:49.3406064Z inflating: build/lib/torch/include/ATen/ops/_coalesced_native.h 2024-08-08T20:48:49.3406962Z inflating: build/lib/torch/include/ATen/ops/_coalesced_ops.h 2024-08-08T20:48:49.3408094Z inflating: build/lib/torch/include/ATen/ops/_compute_linear_combination.h 2024-08-08T20:48:49.3409439Z inflating: build/lib/torch/include/ATen/ops/_compute_linear_combination_cpu_dispatch.h 2024-08-08T20:48:49.3411028Z inflating: build/lib/torch/include/ATen/ops/_compute_linear_combination_cuda_dispatch.h 2024-08-08T20:48:49.3412414Z inflating: build/lib/torch/include/ATen/ops/_compute_linear_combination_native.h 2024-08-08T20:48:49.3413724Z inflating: build/lib/torch/include/ATen/ops/_compute_linear_combination_ops.h 2024-08-08T20:48:49.3414833Z inflating: build/lib/torch/include/ATen/ops/_conj.h 2024-08-08T20:48:49.3416012Z inflating: build/lib/torch/include/ATen/ops/_conj_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3417210Z inflating: build/lib/torch/include/ATen/ops/_conj_copy.h 2024-08-08T20:48:49.3418746Z inflating: build/lib/torch/include/ATen/ops/_conj_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3420420Z inflating: build/lib/torch/include/ATen/ops/_conj_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.3421895Z inflating: build/lib/torch/include/ATen/ops/_conj_copy_native.h 2024-08-08T20:48:49.3422961Z inflating: build/lib/torch/include/ATen/ops/_conj_copy_ops.h 2024-08-08T20:48:49.3423966Z inflating: build/lib/torch/include/ATen/ops/_conj_native.h 2024-08-08T20:48:49.3424934Z inflating: build/lib/torch/include/ATen/ops/_conj_ops.h 2024-08-08T20:48:49.3425921Z inflating: build/lib/torch/include/ATen/ops/_conj_physical.h 2024-08-08T20:48:49.3427217Z inflating: build/lib/torch/include/ATen/ops/_conj_physical_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3428581Z inflating: build/lib/torch/include/ATen/ops/_conj_physical_native.h 2024-08-08T20:48:49.3429783Z inflating: build/lib/torch/include/ATen/ops/_conj_physical_ops.h 2024-08-08T20:48:49.3430924Z inflating: build/lib/torch/include/ATen/ops/_conv_depthwise2d.h 2024-08-08T20:48:49.3432114Z inflating: build/lib/torch/include/ATen/ops/_conv_depthwise2d_cuda_dispatch.h 2024-08-08T20:48:49.3433599Z inflating: build/lib/torch/include/ATen/ops/_conv_depthwise2d_native.h 2024-08-08T20:48:49.3434793Z inflating: build/lib/torch/include/ATen/ops/_conv_depthwise2d_ops.h 2024-08-08T20:48:49.3436001Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_coo_to_csr.h 2024-08-08T20:48:49.3437789Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_coo_to_csr_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.3439740Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_coo_to_csr_cpu_dispatch.h 2024-08-08T20:48:49.3441313Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_coo_to_csr_cuda_dispatch.h 2024-08-08T20:48:49.3442780Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_coo_to_csr_meta.h 2024-08-08T20:48:49.3444355Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_coo_to_csr_meta_dispatch.h 2024-08-08T20:48:49.3445890Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_coo_to_csr_native.h 2024-08-08T20:48:49.3447325Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_coo_to_csr_ops.h 2024-08-08T20:48:49.3448632Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_csr_to_coo.h 2024-08-08T20:48:49.3450287Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_csr_to_coo_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.3452174Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_csr_to_coo_cpu_dispatch.h 2024-08-08T20:48:49.3453820Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_csr_to_coo_cuda_dispatch.h 2024-08-08T20:48:49.3455386Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_csr_to_coo_meta.h 2024-08-08T20:48:49.3456822Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_csr_to_coo_meta_dispatch.h 2024-08-08T20:48:49.3458845Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_csr_to_coo_native.h 2024-08-08T20:48:49.3460408Z inflating: build/lib/torch/include/ATen/ops/_convert_indices_from_csr_to_coo_ops.h 2024-08-08T20:48:49.3461707Z inflating: build/lib/torch/include/ATen/ops/_convert_weight_to_int4pack.h 2024-08-08T20:48:49.3463084Z inflating: build/lib/torch/include/ATen/ops/_convert_weight_to_int4pack_cpu_dispatch.h 2024-08-08T20:48:49.3464589Z inflating: build/lib/torch/include/ATen/ops/_convert_weight_to_int4pack_cuda_dispatch.h 2024-08-08T20:48:49.3466118Z inflating: build/lib/torch/include/ATen/ops/_convert_weight_to_int4pack_native.h 2024-08-08T20:48:49.3467442Z inflating: build/lib/torch/include/ATen/ops/_convert_weight_to_int4pack_ops.h 2024-08-08T20:48:49.3468588Z inflating: build/lib/torch/include/ATen/ops/_convolution.h 2024-08-08T20:48:49.3470059Z inflating: build/lib/torch/include/ATen/ops/_convolution_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3471794Z inflating: build/lib/torch/include/ATen/ops/_convolution_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3473192Z inflating: build/lib/torch/include/ATen/ops/_convolution_double_backward.h 2024-08-08T20:48:49.3474855Z inflating: build/lib/torch/include/ATen/ops/_convolution_double_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3476395Z inflating: build/lib/torch/include/ATen/ops/_convolution_double_backward_native.h 2024-08-08T20:48:49.3477714Z inflating: build/lib/torch/include/ATen/ops/_convolution_double_backward_ops.h 2024-08-08T20:48:49.3478931Z inflating: build/lib/torch/include/ATen/ops/_convolution_mode.h 2024-08-08T20:48:49.3480391Z inflating: build/lib/torch/include/ATen/ops/_convolution_mode_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3481801Z inflating: build/lib/torch/include/ATen/ops/_convolution_mode_native.h 2024-08-08T20:48:49.3482977Z inflating: build/lib/torch/include/ATen/ops/_convolution_mode_ops.h 2024-08-08T20:48:49.3484195Z inflating: build/lib/torch/include/ATen/ops/_convolution_native.h 2024-08-08T20:48:49.3485331Z inflating: build/lib/torch/include/ATen/ops/_convolution_ops.h 2024-08-08T20:48:49.3486368Z inflating: build/lib/torch/include/ATen/ops/_copy_from.h 2024-08-08T20:48:49.3487429Z inflating: build/lib/torch/include/ATen/ops/_copy_from_and_resize.h 2024-08-08T20:48:49.3488983Z inflating: build/lib/torch/include/ATen/ops/_copy_from_and_resize_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3490511Z inflating: build/lib/torch/include/ATen/ops/_copy_from_and_resize_native.h 2024-08-08T20:48:49.3491801Z inflating: build/lib/torch/include/ATen/ops/_copy_from_and_resize_ops.h 2024-08-08T20:48:49.3493201Z inflating: build/lib/torch/include/ATen/ops/_copy_from_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3494512Z inflating: build/lib/torch/include/ATen/ops/_copy_from_native.h 2024-08-08T20:48:49.3495565Z inflating: build/lib/torch/include/ATen/ops/_copy_from_ops.h 2024-08-08T20:48:49.3496586Z inflating: build/lib/torch/include/ATen/ops/_cslt_compress.h 2024-08-08T20:48:49.3497711Z inflating: build/lib/torch/include/ATen/ops/_cslt_compress_cuda_dispatch.h 2024-08-08T20:48:49.3498921Z inflating: build/lib/torch/include/ATen/ops/_cslt_compress_native.h 2024-08-08T20:48:49.3500026Z inflating: build/lib/torch/include/ATen/ops/_cslt_compress_ops.h 2024-08-08T20:48:49.3501108Z inflating: build/lib/torch/include/ATen/ops/_cslt_sparse_mm.h 2024-08-08T20:48:49.3502375Z inflating: build/lib/torch/include/ATen/ops/_cslt_sparse_mm_cuda_dispatch.h 2024-08-08T20:48:49.3503614Z inflating: build/lib/torch/include/ATen/ops/_cslt_sparse_mm_native.h 2024-08-08T20:48:49.3504731Z inflating: build/lib/torch/include/ATen/ops/_cslt_sparse_mm_ops.h 2024-08-08T20:48:49.3505841Z inflating: build/lib/torch/include/ATen/ops/_cslt_sparse_mm_search.h 2024-08-08T20:48:49.3507106Z inflating: build/lib/torch/include/ATen/ops/_cslt_sparse_mm_search_cuda_dispatch.h 2024-08-08T20:48:49.3508604Z inflating: build/lib/torch/include/ATen/ops/_cslt_sparse_mm_search_native.h 2024-08-08T20:48:49.3509881Z inflating: build/lib/torch/include/ATen/ops/_cslt_sparse_mm_search_ops.h 2024-08-08T20:48:49.3510968Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss.h 2024-08-08T20:48:49.3511988Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_backward.h 2024-08-08T20:48:49.3513358Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3514921Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_backward_cpu_dispatch.h 2024-08-08T20:48:49.3516070Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_backward_cuda_dispatch.h 2024-08-08T20:48:49.3517490Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_backward_native.h 2024-08-08T20:48:49.3518774Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_backward_ops.h 2024-08-08T20:48:49.3520111Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3521459Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_cpu_dispatch.h 2024-08-08T20:48:49.3522662Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_cuda_dispatch.h 2024-08-08T20:48:49.3523860Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_meta_dispatch.h 2024-08-08T20:48:49.3524869Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_native.h 2024-08-08T20:48:49.3525896Z inflating: build/lib/torch/include/ATen/ops/_ctc_loss_ops.h 2024-08-08T20:48:49.3526945Z inflating: build/lib/torch/include/ATen/ops/_cudnn_ctc_loss.h 2024-08-08T20:48:49.3528384Z inflating: build/lib/torch/include/ATen/ops/_cudnn_ctc_loss_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3529821Z inflating: build/lib/torch/include/ATen/ops/_cudnn_ctc_loss_cuda_dispatch.h 2024-08-08T20:48:49.3531028Z inflating: build/lib/torch/include/ATen/ops/_cudnn_ctc_loss_native.h 2024-08-08T20:48:49.3532423Z inflating: build/lib/torch/include/ATen/ops/_cudnn_ctc_loss_ops.h 2024-08-08T20:48:49.3533607Z inflating: build/lib/torch/include/ATen/ops/_cudnn_init_dropout_state.h 2024-08-08T20:48:49.3535041Z inflating: build/lib/torch/include/ATen/ops/_cudnn_init_dropout_state_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3536762Z inflating: build/lib/torch/include/ATen/ops/_cudnn_init_dropout_state_cuda_dispatch.h 2024-08-08T20:48:49.3538263Z inflating: build/lib/torch/include/ATen/ops/_cudnn_init_dropout_state_native.h 2024-08-08T20:48:49.3539543Z inflating: build/lib/torch/include/ATen/ops/_cudnn_init_dropout_state_ops.h 2024-08-08T20:48:49.3540664Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn.h 2024-08-08T20:48:49.3541687Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_backward.h 2024-08-08T20:48:49.3543224Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3544672Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_backward_cuda_dispatch.h 2024-08-08T20:48:49.3546033Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_backward_native.h 2024-08-08T20:48:49.3547284Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_backward_ops.h 2024-08-08T20:48:49.3548620Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3550042Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_cuda_dispatch.h 2024-08-08T20:48:49.3551315Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_flatten_weight.h 2024-08-08T20:48:49.3552750Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_flatten_weight_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3554442Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_flatten_weight_cuda_dispatch.h 2024-08-08T20:48:49.3555815Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_flatten_weight_native.h 2024-08-08T20:48:49.3557252Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_flatten_weight_ops.h 2024-08-08T20:48:49.3558712Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_native.h 2024-08-08T20:48:49.3575501Z inflating: build/lib/torch/include/ATen/ops/_cudnn_rnn_ops.h 2024-08-08T20:48:49.3576905Z inflating: build/lib/torch/include/ATen/ops/_cufft_clear_plan_cache.h 2024-08-08T20:48:49.3578166Z inflating: build/lib/torch/include/ATen/ops/_cufft_clear_plan_cache_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3579480Z inflating: build/lib/torch/include/ATen/ops/_cufft_clear_plan_cache_native.h 2024-08-08T20:48:49.3580545Z inflating: build/lib/torch/include/ATen/ops/_cufft_clear_plan_cache_ops.h 2024-08-08T20:48:49.3581625Z inflating: build/lib/torch/include/ATen/ops/_cufft_get_plan_cache_max_size.h 2024-08-08T20:48:49.3583245Z inflating: build/lib/torch/include/ATen/ops/_cufft_get_plan_cache_max_size_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3584630Z inflating: build/lib/torch/include/ATen/ops/_cufft_get_plan_cache_max_size_native.h 2024-08-08T20:48:49.3585806Z inflating: build/lib/torch/include/ATen/ops/_cufft_get_plan_cache_max_size_ops.h 2024-08-08T20:48:49.3586910Z inflating: build/lib/torch/include/ATen/ops/_cufft_get_plan_cache_size.h 2024-08-08T20:48:49.3588209Z inflating: build/lib/torch/include/ATen/ops/_cufft_get_plan_cache_size_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3589584Z inflating: build/lib/torch/include/ATen/ops/_cufft_get_plan_cache_size_native.h 2024-08-08T20:48:49.3590708Z inflating: build/lib/torch/include/ATen/ops/_cufft_get_plan_cache_size_ops.h 2024-08-08T20:48:49.3591807Z inflating: build/lib/torch/include/ATen/ops/_cufft_set_plan_cache_max_size.h 2024-08-08T20:48:49.3593138Z inflating: build/lib/torch/include/ATen/ops/_cufft_set_plan_cache_max_size_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3594551Z inflating: build/lib/torch/include/ATen/ops/_cufft_set_plan_cache_max_size_native.h 2024-08-08T20:48:49.3595845Z inflating: build/lib/torch/include/ATen/ops/_cufft_set_plan_cache_max_size_ops.h 2024-08-08T20:48:49.3596879Z inflating: build/lib/torch/include/ATen/ops/_cummax_helper.h 2024-08-08T20:48:49.3597821Z inflating: build/lib/torch/include/ATen/ops/_cummax_helper_cpu_dispatch.h 2024-08-08T20:48:49.3598891Z inflating: build/lib/torch/include/ATen/ops/_cummax_helper_cuda_dispatch.h 2024-08-08T20:48:49.3599913Z inflating: build/lib/torch/include/ATen/ops/_cummax_helper_native.h 2024-08-08T20:48:49.3600838Z inflating: build/lib/torch/include/ATen/ops/_cummax_helper_ops.h 2024-08-08T20:48:49.3601725Z inflating: build/lib/torch/include/ATen/ops/_cummin_helper.h 2024-08-08T20:48:49.3602677Z inflating: build/lib/torch/include/ATen/ops/_cummin_helper_cpu_dispatch.h 2024-08-08T20:48:49.3603737Z inflating: build/lib/torch/include/ATen/ops/_cummin_helper_cuda_dispatch.h 2024-08-08T20:48:49.3604759Z inflating: build/lib/torch/include/ATen/ops/_cummin_helper_native.h 2024-08-08T20:48:49.3605707Z inflating: build/lib/torch/include/ATen/ops/_cummin_helper_ops.h 2024-08-08T20:48:49.3606699Z inflating: build/lib/torch/include/ATen/ops/_debug_has_internal_overlap.h 2024-08-08T20:48:49.3607997Z inflating: build/lib/torch/include/ATen/ops/_debug_has_internal_overlap_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3609365Z inflating: build/lib/torch/include/ATen/ops/_debug_has_internal_overlap_native.h 2024-08-08T20:48:49.3610507Z inflating: build/lib/torch/include/ATen/ops/_debug_has_internal_overlap_ops.h 2024-08-08T20:48:49.3611417Z inflating: build/lib/torch/include/ATen/ops/_dimI.h 2024-08-08T20:48:49.3612200Z inflating: build/lib/torch/include/ATen/ops/_dimI_native.h 2024-08-08T20:48:49.3613004Z inflating: build/lib/torch/include/ATen/ops/_dimI_ops.h 2024-08-08T20:48:49.3613746Z inflating: build/lib/torch/include/ATen/ops/_dimV.h 2024-08-08T20:48:49.3614518Z inflating: build/lib/torch/include/ATen/ops/_dimV_native.h 2024-08-08T20:48:49.3615321Z inflating: build/lib/torch/include/ATen/ops/_dimV_ops.h 2024-08-08T20:48:49.3616094Z inflating: build/lib/torch/include/ATen/ops/_dim_arange.h 2024-08-08T20:48:49.3617165Z inflating: build/lib/torch/include/ATen/ops/_dim_arange_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3618303Z inflating: build/lib/torch/include/ATen/ops/_dim_arange_native.h 2024-08-08T20:48:49.3619198Z inflating: build/lib/torch/include/ATen/ops/_dim_arange_ops.h 2024-08-08T20:48:49.3620049Z inflating: build/lib/torch/include/ATen/ops/_dirichlet_grad.h 2024-08-08T20:48:49.3621183Z inflating: build/lib/torch/include/ATen/ops/_dirichlet_grad_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3622431Z inflating: build/lib/torch/include/ATen/ops/_dirichlet_grad_cpu_dispatch.h 2024-08-08T20:48:49.3623627Z inflating: build/lib/torch/include/ATen/ops/_dirichlet_grad_cuda_dispatch.h 2024-08-08T20:48:49.3624660Z inflating: build/lib/torch/include/ATen/ops/_dirichlet_grad_native.h 2024-08-08T20:48:49.3625621Z inflating: build/lib/torch/include/ATen/ops/_dirichlet_grad_ops.h 2024-08-08T20:48:49.3626605Z inflating: build/lib/torch/include/ATen/ops/_efficient_attention_backward.h 2024-08-08T20:48:49.3627795Z inflating: build/lib/torch/include/ATen/ops/_efficient_attention_backward_cuda_dispatch.h 2024-08-08T20:48:49.3629026Z inflating: build/lib/torch/include/ATen/ops/_efficient_attention_backward_native.h 2024-08-08T20:48:49.3630195Z inflating: build/lib/torch/include/ATen/ops/_efficient_attention_backward_ops.h 2024-08-08T20:48:49.3631288Z inflating: build/lib/torch/include/ATen/ops/_efficient_attention_forward.h 2024-08-08T20:48:49.3632463Z inflating: build/lib/torch/include/ATen/ops/_efficient_attention_forward_cuda_dispatch.h 2024-08-08T20:48:49.3633687Z inflating: build/lib/torch/include/ATen/ops/_efficient_attention_forward_native.h 2024-08-08T20:48:49.3634849Z inflating: build/lib/torch/include/ATen/ops/_efficient_attention_forward_ops.h 2024-08-08T20:48:49.3635988Z inflating: build/lib/torch/include/ATen/ops/_efficientzerotensor.h 2024-08-08T20:48:49.3637238Z inflating: build/lib/torch/include/ATen/ops/_efficientzerotensor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3638550Z inflating: build/lib/torch/include/ATen/ops/_efficientzerotensor_cpu_dispatch.h 2024-08-08T20:48:49.3639721Z inflating: build/lib/torch/include/ATen/ops/_efficientzerotensor_cuda_dispatch.h 2024-08-08T20:48:49.3640902Z inflating: build/lib/torch/include/ATen/ops/_efficientzerotensor_meta_dispatch.h 2024-08-08T20:48:49.3642025Z inflating: build/lib/torch/include/ATen/ops/_efficientzerotensor_native.h 2024-08-08T20:48:49.3643056Z inflating: build/lib/torch/include/ATen/ops/_efficientzerotensor_ops.h 2024-08-08T20:48:49.3644001Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag.h 2024-08-08T20:48:49.3644937Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_backward.h 2024-08-08T20:48:49.3646020Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_backward_cpu_dispatch.h 2024-08-08T20:48:49.3647219Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_backward_cuda_dispatch.h 2024-08-08T20:48:49.3648370Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_backward_native.h 2024-08-08T20:48:49.3649436Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_backward_ops.h 2024-08-08T20:48:49.3650658Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3651884Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_cpu_dispatch.h 2024-08-08T20:48:49.3652946Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_cuda_dispatch.h 2024-08-08T20:48:49.3654003Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_dense_backward.h 2024-08-08T20:48:49.3655353Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_dense_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3656789Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_dense_backward_cpu_dispatch.h 2024-08-08T20:48:49.3658050Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_dense_backward_cuda_dispatch.h 2024-08-08T20:48:49.3659821Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_dense_backward_native.h 2024-08-08T20:48:49.3661354Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_dense_backward_ops.h 2024-08-08T20:48:49.3662549Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_forward_only.h 2024-08-08T20:48:49.3663841Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_forward_only_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3665249Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_forward_only_cpu_dispatch.h 2024-08-08T20:48:49.3666671Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_forward_only_cuda_dispatch.h 2024-08-08T20:48:49.3667870Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_forward_only_native.h 2024-08-08T20:48:49.3669010Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_forward_only_ops.h 2024-08-08T20:48:49.3670055Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_native.h 2024-08-08T20:48:49.3671000Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_ops.h 2024-08-08T20:48:49.3672185Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_per_sample_weights_backward.h 2024-08-08T20:48:49.3673695Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_per_sample_weights_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3675292Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_per_sample_weights_backward_cpu_dispatch.h 2024-08-08T20:48:49.3676739Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_per_sample_weights_backward_cuda_dispatch.h 2024-08-08T20:48:49.3678153Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_per_sample_weights_backward_native.h 2024-08-08T20:48:49.3679588Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_per_sample_weights_backward_ops.h 2024-08-08T20:48:49.3680836Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_sparse_backward.h 2024-08-08T20:48:49.3682195Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_sparse_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3683576Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_sparse_backward_native.h 2024-08-08T20:48:49.3684761Z inflating: build/lib/torch/include/ATen/ops/_embedding_bag_sparse_backward_ops.h 2024-08-08T20:48:49.3685845Z inflating: build/lib/torch/include/ATen/ops/_empty_affine_quantized.h 2024-08-08T20:48:49.3687099Z inflating: build/lib/torch/include/ATen/ops/_empty_affine_quantized_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3688435Z inflating: build/lib/torch/include/ATen/ops/_empty_affine_quantized_cpu_dispatch.h 2024-08-08T20:48:49.3689633Z inflating: build/lib/torch/include/ATen/ops/_empty_affine_quantized_native.h 2024-08-08T20:48:49.3690722Z inflating: build/lib/torch/include/ATen/ops/_empty_affine_quantized_ops.h 2024-08-08T20:48:49.3691828Z inflating: build/lib/torch/include/ATen/ops/_empty_per_channel_affine_quantized.h 2024-08-08T20:48:49.3693255Z inflating: build/lib/torch/include/ATen/ops/_empty_per_channel_affine_quantized_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3694756Z inflating: build/lib/torch/include/ATen/ops/_empty_per_channel_affine_quantized_cpu_dispatch.h 2024-08-08T20:48:49.3696055Z inflating: build/lib/torch/include/ATen/ops/_empty_per_channel_affine_quantized_native.h 2024-08-08T20:48:49.3697280Z inflating: build/lib/torch/include/ATen/ops/_empty_per_channel_affine_quantized_ops.h 2024-08-08T20:48:49.3698333Z inflating: build/lib/torch/include/ATen/ops/_euclidean_dist.h 2024-08-08T20:48:49.3699482Z inflating: build/lib/torch/include/ATen/ops/_euclidean_dist_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3700671Z inflating: build/lib/torch/include/ATen/ops/_euclidean_dist_native.h 2024-08-08T20:48:49.3701630Z inflating: build/lib/torch/include/ATen/ops/_euclidean_dist_ops.h 2024-08-08T20:48:49.3702733Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine.h 2024-08-08T20:48:49.3704076Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_backward.h 2024-08-08T20:48:49.3705548Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_backward_cpu_dispatch.h 2024-08-08T20:48:49.3707130Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_backward_cuda_dispatch.h 2024-08-08T20:48:49.3708662Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_backward_native.h 2024-08-08T20:48:49.3710203Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_backward_ops.h 2024-08-08T20:48:49.3711813Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3713428Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_cpu_dispatch.h 2024-08-08T20:48:49.3714885Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_cuda_dispatch.h 2024-08-08T20:48:49.3716280Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_native.h 2024-08-08T20:48:49.3717633Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_channel_affine_ops.h 2024-08-08T20:48:49.3718926Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine.h 2024-08-08T20:48:49.3720244Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_backward.h 2024-08-08T20:48:49.3721809Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_backward_cpu_dispatch.h 2024-08-08T20:48:49.3723388Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_backward_cuda_dispatch.h 2024-08-08T20:48:49.3724903Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_backward_native.h 2024-08-08T20:48:49.3726337Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_backward_ops.h 2024-08-08T20:48:49.3727926Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3729515Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_cpu_dispatch.h 2024-08-08T20:48:49.3730959Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_cuda_dispatch.h 2024-08-08T20:48:49.3732353Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_native.h 2024-08-08T20:48:49.3733694Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_learnable_per_tensor_affine_ops.h 2024-08-08T20:48:49.3735094Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_per_tensor_affine_cachemask_tensor_qparams.h 2024-08-08T20:48:49.3736829Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_per_tensor_affine_cachemask_tensor_qparams_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3738619Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_per_tensor_affine_cachemask_tensor_qparams_cpu_dispatch.h 2024-08-08T20:48:49.3740257Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_per_tensor_affine_cachemask_tensor_qparams_cuda_dispatch.h 2024-08-08T20:48:49.3741861Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_per_tensor_affine_cachemask_tensor_qparams_native.h 2024-08-08T20:48:49.3743389Z inflating: build/lib/torch/include/ATen/ops/_fake_quantize_per_tensor_affine_cachemask_tensor_qparams_ops.h 2024-08-08T20:48:49.3744543Z inflating: build/lib/torch/include/ATen/ops/_fft_c2c.h 2024-08-08T20:48:49.3745402Z inflating: build/lib/torch/include/ATen/ops/_fft_c2c_cpu_dispatch.h 2024-08-08T20:48:49.3746354Z inflating: build/lib/torch/include/ATen/ops/_fft_c2c_cuda_dispatch.h 2024-08-08T20:48:49.3747275Z inflating: build/lib/torch/include/ATen/ops/_fft_c2c_native.h 2024-08-08T20:48:49.3748112Z inflating: build/lib/torch/include/ATen/ops/_fft_c2c_ops.h 2024-08-08T20:48:49.3748897Z inflating: build/lib/torch/include/ATen/ops/_fft_c2r.h 2024-08-08T20:48:49.3749731Z inflating: build/lib/torch/include/ATen/ops/_fft_c2r_cpu_dispatch.h 2024-08-08T20:48:49.3750701Z inflating: build/lib/torch/include/ATen/ops/_fft_c2r_cuda_dispatch.h 2024-08-08T20:48:49.3751623Z inflating: build/lib/torch/include/ATen/ops/_fft_c2r_native.h 2024-08-08T20:48:49.3752558Z inflating: build/lib/torch/include/ATen/ops/_fft_c2r_ops.h 2024-08-08T20:48:49.3753340Z inflating: build/lib/torch/include/ATen/ops/_fft_r2c.h 2024-08-08T20:48:49.3754202Z inflating: build/lib/torch/include/ATen/ops/_fft_r2c_cpu_dispatch.h 2024-08-08T20:48:49.3755157Z inflating: build/lib/torch/include/ATen/ops/_fft_r2c_cuda_dispatch.h 2024-08-08T20:48:49.3756078Z inflating: build/lib/torch/include/ATen/ops/_fft_r2c_native.h 2024-08-08T20:48:49.3756919Z inflating: build/lib/torch/include/ATen/ops/_fft_r2c_ops.h 2024-08-08T20:48:49.3757821Z inflating: build/lib/torch/include/ATen/ops/_fill_mem_eff_dropout_mask.h 2024-08-08T20:48:49.3759229Z inflating: build/lib/torch/include/ATen/ops/_fill_mem_eff_dropout_mask_cuda_dispatch.h 2024-08-08T20:48:49.3760469Z inflating: build/lib/torch/include/ATen/ops/_fill_mem_eff_dropout_mask_meta_dispatch.h 2024-08-08T20:48:49.3761655Z inflating: build/lib/torch/include/ATen/ops/_fill_mem_eff_dropout_mask_native.h 2024-08-08T20:48:49.3762769Z inflating: build/lib/torch/include/ATen/ops/_fill_mem_eff_dropout_mask_ops.h 2024-08-08T20:48:49.3763984Z inflating: build/lib/torch/include/ATen/ops/_flash_attention_backward.h 2024-08-08T20:48:49.3765137Z inflating: build/lib/torch/include/ATen/ops/_flash_attention_backward_cuda_dispatch.h 2024-08-08T20:48:49.3766301Z inflating: build/lib/torch/include/ATen/ops/_flash_attention_backward_native.h 2024-08-08T20:48:49.3767406Z inflating: build/lib/torch/include/ATen/ops/_flash_attention_backward_ops.h 2024-08-08T20:48:49.3768458Z inflating: build/lib/torch/include/ATen/ops/_flash_attention_forward.h 2024-08-08T20:48:49.3769557Z inflating: build/lib/torch/include/ATen/ops/_flash_attention_forward_cuda_dispatch.h 2024-08-08T20:48:49.3770721Z inflating: build/lib/torch/include/ATen/ops/_flash_attention_forward_native.h 2024-08-08T20:48:49.3771804Z inflating: build/lib/torch/include/ATen/ops/_flash_attention_forward_ops.h 2024-08-08T20:48:49.3772730Z inflating: build/lib/torch/include/ATen/ops/_foobar.h 2024-08-08T20:48:49.3773733Z inflating: build/lib/torch/include/ATen/ops/_foobar_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3774863Z inflating: build/lib/torch/include/ATen/ops/_foobar_cpu_dispatch.h 2024-08-08T20:48:49.3775769Z inflating: build/lib/torch/include/ATen/ops/_foobar_native.h 2024-08-08T20:48:49.3776575Z inflating: build/lib/torch/include/ATen/ops/_foobar_ops.h 2024-08-08T20:48:49.3777379Z inflating: build/lib/torch/include/ATen/ops/_foreach_abs.h 2024-08-08T20:48:49.3778466Z inflating: build/lib/torch/include/ATen/ops/_foreach_abs_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3779655Z inflating: build/lib/torch/include/ATen/ops/_foreach_abs_cuda_dispatch.h 2024-08-08T20:48:49.3780637Z inflating: build/lib/torch/include/ATen/ops/_foreach_abs_native.h 2024-08-08T20:48:49.3781545Z inflating: build/lib/torch/include/ATen/ops/_foreach_abs_ops.h 2024-08-08T20:48:49.3782399Z inflating: build/lib/torch/include/ATen/ops/_foreach_acos.h 2024-08-08T20:48:49.3783514Z inflating: build/lib/torch/include/ATen/ops/_foreach_acos_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3784735Z inflating: build/lib/torch/include/ATen/ops/_foreach_acos_cuda_dispatch.h 2024-08-08T20:48:49.3785739Z inflating: build/lib/torch/include/ATen/ops/_foreach_acos_native.h 2024-08-08T20:48:49.3786642Z inflating: build/lib/torch/include/ATen/ops/_foreach_acos_ops.h 2024-08-08T20:48:49.3787501Z inflating: build/lib/torch/include/ATen/ops/_foreach_add.h 2024-08-08T20:48:49.3788592Z inflating: build/lib/torch/include/ATen/ops/_foreach_add_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3789820Z inflating: build/lib/torch/include/ATen/ops/_foreach_add_cuda_dispatch.h 2024-08-08T20:48:49.3790807Z inflating: build/lib/torch/include/ATen/ops/_foreach_add_native.h 2024-08-08T20:48:49.3791715Z inflating: build/lib/torch/include/ATen/ops/_foreach_add_ops.h 2024-08-08T20:48:49.3792719Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcdiv.h 2024-08-08T20:48:49.3793872Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcdiv_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3795144Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcdiv_cuda_dispatch.h 2024-08-08T20:48:49.3796200Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcdiv_native.h 2024-08-08T20:48:49.3797148Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcdiv_ops.h 2024-08-08T20:48:49.3798058Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcmul.h 2024-08-08T20:48:49.3799215Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcmul_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3800459Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcmul_cuda_dispatch.h 2024-08-08T20:48:49.3801500Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcmul_native.h 2024-08-08T20:48:49.3802479Z inflating: build/lib/torch/include/ATen/ops/_foreach_addcmul_ops.h 2024-08-08T20:48:49.3803358Z inflating: build/lib/torch/include/ATen/ops/_foreach_asin.h 2024-08-08T20:48:49.3804547Z inflating: build/lib/torch/include/ATen/ops/_foreach_asin_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3805787Z inflating: build/lib/torch/include/ATen/ops/_foreach_asin_cuda_dispatch.h 2024-08-08T20:48:49.3806769Z inflating: build/lib/torch/include/ATen/ops/_foreach_asin_native.h 2024-08-08T20:48:49.3807670Z inflating: build/lib/torch/include/ATen/ops/_foreach_asin_ops.h 2024-08-08T20:48:49.3808550Z inflating: build/lib/torch/include/ATen/ops/_foreach_atan.h 2024-08-08T20:48:49.3809661Z inflating: build/lib/torch/include/ATen/ops/_foreach_atan_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3810858Z inflating: build/lib/torch/include/ATen/ops/_foreach_atan_cuda_dispatch.h 2024-08-08T20:48:49.3811850Z inflating: build/lib/torch/include/ATen/ops/_foreach_atan_native.h 2024-08-08T20:48:49.3812780Z inflating: build/lib/torch/include/ATen/ops/_foreach_atan_ops.h 2024-08-08T20:48:49.3813642Z inflating: build/lib/torch/include/ATen/ops/_foreach_ceil.h 2024-08-08T20:48:49.3814753Z inflating: build/lib/torch/include/ATen/ops/_foreach_ceil_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3815981Z inflating: build/lib/torch/include/ATen/ops/_foreach_ceil_cuda_dispatch.h 2024-08-08T20:48:49.3816975Z inflating: build/lib/torch/include/ATen/ops/_foreach_ceil_native.h 2024-08-08T20:48:49.3817878Z inflating: build/lib/torch/include/ATen/ops/_foreach_ceil_ops.h 2024-08-08T20:48:49.3818779Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_max.h 2024-08-08T20:48:49.3819956Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_max_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3821232Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_max_cuda_dispatch.h 2024-08-08T20:48:49.3822312Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_max_native.h 2024-08-08T20:48:49.3823324Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_max_ops.h 2024-08-08T20:48:49.3824255Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_min.h 2024-08-08T20:48:49.3825432Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_min_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3826714Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_min_cuda_dispatch.h 2024-08-08T20:48:49.3827794Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_min_native.h 2024-08-08T20:48:49.3828781Z inflating: build/lib/torch/include/ATen/ops/_foreach_clamp_min_ops.h 2024-08-08T20:48:49.3829693Z inflating: build/lib/torch/include/ATen/ops/_foreach_copy.h 2024-08-08T20:48:49.3830799Z inflating: build/lib/torch/include/ATen/ops/_foreach_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3831994Z inflating: build/lib/torch/include/ATen/ops/_foreach_copy_cuda_dispatch.h 2024-08-08T20:48:49.3833100Z inflating: build/lib/torch/include/ATen/ops/_foreach_copy_native.h 2024-08-08T20:48:49.3834023Z inflating: build/lib/torch/include/ATen/ops/_foreach_copy_ops.h 2024-08-08T20:48:49.3834869Z inflating: build/lib/torch/include/ATen/ops/_foreach_cos.h 2024-08-08T20:48:49.3835961Z inflating: build/lib/torch/include/ATen/ops/_foreach_cos_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3837158Z inflating: build/lib/torch/include/ATen/ops/_foreach_cos_cuda_dispatch.h 2024-08-08T20:48:49.3838139Z inflating: build/lib/torch/include/ATen/ops/_foreach_cos_native.h 2024-08-08T20:48:49.3839028Z inflating: build/lib/torch/include/ATen/ops/_foreach_cos_ops.h 2024-08-08T20:48:49.3839888Z inflating: build/lib/torch/include/ATen/ops/_foreach_cosh.h 2024-08-08T20:48:49.3840992Z inflating: build/lib/torch/include/ATen/ops/_foreach_cosh_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3842182Z inflating: build/lib/torch/include/ATen/ops/_foreach_cosh_cuda_dispatch.h 2024-08-08T20:48:49.3843186Z inflating: build/lib/torch/include/ATen/ops/_foreach_cosh_native.h 2024-08-08T20:48:49.3844199Z inflating: build/lib/torch/include/ATen/ops/_foreach_cosh_ops.h 2024-08-08T20:48:49.3845059Z inflating: build/lib/torch/include/ATen/ops/_foreach_div.h 2024-08-08T20:48:49.3846138Z inflating: build/lib/torch/include/ATen/ops/_foreach_div_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3847349Z inflating: build/lib/torch/include/ATen/ops/_foreach_div_cuda_dispatch.h 2024-08-08T20:48:49.3848330Z inflating: build/lib/torch/include/ATen/ops/_foreach_div_native.h 2024-08-08T20:48:49.3849215Z inflating: build/lib/torch/include/ATen/ops/_foreach_div_ops.h 2024-08-08T20:48:49.3850063Z inflating: build/lib/torch/include/ATen/ops/_foreach_erf.h 2024-08-08T20:48:49.3851156Z inflating: build/lib/torch/include/ATen/ops/_foreach_erf_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3852343Z inflating: build/lib/torch/include/ATen/ops/_foreach_erf_cuda_dispatch.h 2024-08-08T20:48:49.3853338Z inflating: build/lib/torch/include/ATen/ops/_foreach_erf_native.h 2024-08-08T20:48:49.3854245Z inflating: build/lib/torch/include/ATen/ops/_foreach_erf_ops.h 2024-08-08T20:48:49.3855094Z inflating: build/lib/torch/include/ATen/ops/_foreach_erfc.h 2024-08-08T20:48:49.3856291Z inflating: build/lib/torch/include/ATen/ops/_foreach_erfc_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3857489Z inflating: build/lib/torch/include/ATen/ops/_foreach_erfc_cuda_dispatch.h 2024-08-08T20:48:49.3858751Z inflating: build/lib/torch/include/ATen/ops/_foreach_erfc_native.h 2024-08-08T20:48:49.3859676Z inflating: build/lib/torch/include/ATen/ops/_foreach_erfc_ops.h 2024-08-08T20:48:49.3860519Z inflating: build/lib/torch/include/ATen/ops/_foreach_exp.h 2024-08-08T20:48:49.3861609Z inflating: build/lib/torch/include/ATen/ops/_foreach_exp_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3862985Z inflating: build/lib/torch/include/ATen/ops/_foreach_exp_cuda_dispatch.h 2024-08-08T20:48:49.3863987Z inflating: build/lib/torch/include/ATen/ops/_foreach_exp_native.h 2024-08-08T20:48:49.3864881Z inflating: build/lib/torch/include/ATen/ops/_foreach_exp_ops.h 2024-08-08T20:48:49.3865747Z inflating: build/lib/torch/include/ATen/ops/_foreach_expm1.h 2024-08-08T20:48:49.3866868Z inflating: build/lib/torch/include/ATen/ops/_foreach_expm1_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3868076Z inflating: build/lib/torch/include/ATen/ops/_foreach_expm1_cuda_dispatch.h 2024-08-08T20:48:49.3869094Z inflating: build/lib/torch/include/ATen/ops/_foreach_expm1_native.h 2024-08-08T20:48:49.3870035Z inflating: build/lib/torch/include/ATen/ops/_foreach_expm1_ops.h 2024-08-08T20:48:49.3870903Z inflating: build/lib/torch/include/ATen/ops/_foreach_floor.h 2024-08-08T20:48:49.3872031Z inflating: build/lib/torch/include/ATen/ops/_foreach_floor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3873420Z inflating: build/lib/torch/include/ATen/ops/_foreach_floor_cuda_dispatch.h 2024-08-08T20:48:49.3874441Z inflating: build/lib/torch/include/ATen/ops/_foreach_floor_native.h 2024-08-08T20:48:49.3875369Z inflating: build/lib/torch/include/ATen/ops/_foreach_floor_ops.h 2024-08-08T20:48:49.3876255Z inflating: build/lib/torch/include/ATen/ops/_foreach_frac.h 2024-08-08T20:48:49.3877363Z inflating: build/lib/torch/include/ATen/ops/_foreach_frac_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3878561Z inflating: build/lib/torch/include/ATen/ops/_foreach_frac_cuda_dispatch.h 2024-08-08T20:48:49.3879552Z inflating: build/lib/torch/include/ATen/ops/_foreach_frac_native.h 2024-08-08T20:48:49.3880478Z inflating: build/lib/torch/include/ATen/ops/_foreach_frac_ops.h 2024-08-08T20:48:49.3881336Z inflating: build/lib/torch/include/ATen/ops/_foreach_lerp.h 2024-08-08T20:48:49.3882442Z inflating: build/lib/torch/include/ATen/ops/_foreach_lerp_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3883668Z inflating: build/lib/torch/include/ATen/ops/_foreach_lerp_cuda_dispatch.h 2024-08-08T20:48:49.3884757Z inflating: build/lib/torch/include/ATen/ops/_foreach_lerp_native.h 2024-08-08T20:48:49.3885705Z inflating: build/lib/torch/include/ATen/ops/_foreach_lerp_ops.h 2024-08-08T20:48:49.3886598Z inflating: build/lib/torch/include/ATen/ops/_foreach_lgamma.h 2024-08-08T20:48:49.3887727Z inflating: build/lib/torch/include/ATen/ops/_foreach_lgamma_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3888949Z inflating: build/lib/torch/include/ATen/ops/_foreach_lgamma_cuda_dispatch.h 2024-08-08T20:48:49.3890030Z inflating: build/lib/torch/include/ATen/ops/_foreach_lgamma_native.h 2024-08-08T20:48:49.3890977Z inflating: build/lib/torch/include/ATen/ops/_foreach_lgamma_ops.h 2024-08-08T20:48:49.3891829Z inflating: build/lib/torch/include/ATen/ops/_foreach_log.h 2024-08-08T20:48:49.3892659Z inflating: build/lib/torch/include/ATen/ops/_foreach_log10.h 2024-08-08T20:48:49.3893790Z inflating: build/lib/torch/include/ATen/ops/_foreach_log10_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3895004Z inflating: build/lib/torch/include/ATen/ops/_foreach_log10_cuda_dispatch.h 2024-08-08T20:48:49.3896025Z inflating: build/lib/torch/include/ATen/ops/_foreach_log10_native.h 2024-08-08T20:48:49.3896957Z inflating: build/lib/torch/include/ATen/ops/_foreach_log10_ops.h 2024-08-08T20:48:49.3897837Z inflating: build/lib/torch/include/ATen/ops/_foreach_log1p.h 2024-08-08T20:48:49.3898944Z inflating: build/lib/torch/include/ATen/ops/_foreach_log1p_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3900174Z inflating: build/lib/torch/include/ATen/ops/_foreach_log1p_cuda_dispatch.h 2024-08-08T20:48:49.3901191Z inflating: build/lib/torch/include/ATen/ops/_foreach_log1p_native.h 2024-08-08T20:48:49.3902111Z inflating: build/lib/torch/include/ATen/ops/_foreach_log1p_ops.h 2024-08-08T20:48:49.3902988Z inflating: build/lib/torch/include/ATen/ops/_foreach_log2.h 2024-08-08T20:48:49.3904099Z inflating: build/lib/torch/include/ATen/ops/_foreach_log2_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3905303Z inflating: build/lib/torch/include/ATen/ops/_foreach_log2_cuda_dispatch.h 2024-08-08T20:48:49.3906302Z inflating: build/lib/torch/include/ATen/ops/_foreach_log2_native.h 2024-08-08T20:48:49.3907223Z inflating: build/lib/torch/include/ATen/ops/_foreach_log2_ops.h 2024-08-08T20:48:49.3908349Z inflating: build/lib/torch/include/ATen/ops/_foreach_log_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3909529Z inflating: build/lib/torch/include/ATen/ops/_foreach_log_cuda_dispatch.h 2024-08-08T20:48:49.3910514Z inflating: build/lib/torch/include/ATen/ops/_foreach_log_native.h 2024-08-08T20:48:49.3911415Z inflating: build/lib/torch/include/ATen/ops/_foreach_log_ops.h 2024-08-08T20:48:49.3912247Z inflating: build/lib/torch/include/ATen/ops/_foreach_max.h 2024-08-08T20:48:49.3913434Z inflating: build/lib/torch/include/ATen/ops/_foreach_max_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3914634Z inflating: build/lib/torch/include/ATen/ops/_foreach_max_cuda_dispatch.h 2024-08-08T20:48:49.3915597Z inflating: build/lib/torch/include/ATen/ops/_foreach_max_native.h 2024-08-08T20:48:49.3916502Z inflating: build/lib/torch/include/ATen/ops/_foreach_max_ops.h 2024-08-08T20:48:49.3917381Z inflating: build/lib/torch/include/ATen/ops/_foreach_maximum.h 2024-08-08T20:48:49.3918533Z inflating: build/lib/torch/include/ATen/ops/_foreach_maximum_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3919782Z inflating: build/lib/torch/include/ATen/ops/_foreach_maximum_cuda_dispatch.h 2024-08-08T20:48:49.3920831Z inflating: build/lib/torch/include/ATen/ops/_foreach_maximum_native.h 2024-08-08T20:48:49.3921798Z inflating: build/lib/torch/include/ATen/ops/_foreach_maximum_ops.h 2024-08-08T20:48:49.3922688Z inflating: build/lib/torch/include/ATen/ops/_foreach_minimum.h 2024-08-08T20:48:49.3923845Z inflating: build/lib/torch/include/ATen/ops/_foreach_minimum_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3925197Z inflating: build/lib/torch/include/ATen/ops/_foreach_minimum_cuda_dispatch.h 2024-08-08T20:48:49.3926241Z inflating: build/lib/torch/include/ATen/ops/_foreach_minimum_native.h 2024-08-08T20:48:49.3927211Z inflating: build/lib/torch/include/ATen/ops/_foreach_minimum_ops.h 2024-08-08T20:48:49.3928090Z inflating: build/lib/torch/include/ATen/ops/_foreach_mul.h 2024-08-08T20:48:49.3929167Z inflating: build/lib/torch/include/ATen/ops/_foreach_mul_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3930350Z inflating: build/lib/torch/include/ATen/ops/_foreach_mul_cuda_dispatch.h 2024-08-08T20:48:49.3931325Z inflating: build/lib/torch/include/ATen/ops/_foreach_mul_native.h 2024-08-08T20:48:49.3932226Z inflating: build/lib/torch/include/ATen/ops/_foreach_mul_ops.h 2024-08-08T20:48:49.3932547Z inflating: build/lib/torch/include/ATen/ops/_foreach_neg.h 2024-08-08T20:48:49.3933124Z inflating: build/lib/torch/include/ATen/ops/_foreach_neg_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3933563Z inflating: build/lib/torch/include/ATen/ops/_foreach_neg_cuda_dispatch.h 2024-08-08T20:48:49.3933924Z inflating: build/lib/torch/include/ATen/ops/_foreach_neg_native.h 2024-08-08T20:48:49.3934257Z inflating: build/lib/torch/include/ATen/ops/_foreach_neg_ops.h 2024-08-08T20:48:49.3934600Z inflating: build/lib/torch/include/ATen/ops/_foreach_norm.h 2024-08-08T20:48:49.3935186Z inflating: build/lib/torch/include/ATen/ops/_foreach_norm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3935625Z inflating: build/lib/torch/include/ATen/ops/_foreach_norm_cuda_dispatch.h 2024-08-08T20:48:49.3935993Z inflating: build/lib/torch/include/ATen/ops/_foreach_norm_native.h 2024-08-08T20:48:49.3936338Z inflating: build/lib/torch/include/ATen/ops/_foreach_norm_ops.h 2024-08-08T20:48:49.3936677Z inflating: build/lib/torch/include/ATen/ops/_foreach_pow.h 2024-08-08T20:48:49.3937257Z inflating: build/lib/torch/include/ATen/ops/_foreach_pow_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3937681Z inflating: build/lib/torch/include/ATen/ops/_foreach_pow_cuda_dispatch.h 2024-08-08T20:48:49.3938059Z inflating: build/lib/torch/include/ATen/ops/_foreach_pow_native.h 2024-08-08T20:48:49.3938402Z inflating: build/lib/torch/include/ATen/ops/_foreach_pow_ops.h 2024-08-08T20:48:49.3938780Z inflating: build/lib/torch/include/ATen/ops/_foreach_reciprocal.h 2024-08-08T20:48:49.3939408Z inflating: build/lib/torch/include/ATen/ops/_foreach_reciprocal_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3939876Z inflating: build/lib/torch/include/ATen/ops/_foreach_reciprocal_cuda_dispatch.h 2024-08-08T20:48:49.3940308Z inflating: build/lib/torch/include/ATen/ops/_foreach_reciprocal_native.h 2024-08-08T20:48:49.3940704Z inflating: build/lib/torch/include/ATen/ops/_foreach_reciprocal_ops.h 2024-08-08T20:48:49.3941128Z inflating: build/lib/torch/include/ATen/ops/_foreach_round.h 2024-08-08T20:48:49.3941741Z inflating: build/lib/torch/include/ATen/ops/_foreach_round_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3942176Z inflating: build/lib/torch/include/ATen/ops/_foreach_round_cuda_dispatch.h 2024-08-08T20:48:49.3942556Z inflating: build/lib/torch/include/ATen/ops/_foreach_round_native.h 2024-08-08T20:48:49.3942931Z inflating: build/lib/torch/include/ATen/ops/_foreach_round_ops.h 2024-08-08T20:48:49.3943276Z inflating: build/lib/torch/include/ATen/ops/_foreach_sigmoid.h 2024-08-08T20:48:49.3943896Z inflating: build/lib/torch/include/ATen/ops/_foreach_sigmoid_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3944342Z inflating: build/lib/torch/include/ATen/ops/_foreach_sigmoid_cuda_dispatch.h 2024-08-08T20:48:49.3944733Z inflating: build/lib/torch/include/ATen/ops/_foreach_sigmoid_native.h 2024-08-08T20:48:49.3945126Z inflating: build/lib/torch/include/ATen/ops/_foreach_sigmoid_ops.h 2024-08-08T20:48:49.3945442Z inflating: build/lib/torch/include/ATen/ops/_foreach_sign.h 2024-08-08T20:48:49.3946097Z inflating: build/lib/torch/include/ATen/ops/_foreach_sign_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3946550Z inflating: build/lib/torch/include/ATen/ops/_foreach_sign_cuda_dispatch.h 2024-08-08T20:48:49.3946912Z inflating: build/lib/torch/include/ATen/ops/_foreach_sign_native.h 2024-08-08T20:48:49.3947278Z inflating: build/lib/torch/include/ATen/ops/_foreach_sign_ops.h 2024-08-08T20:48:49.3947585Z inflating: build/lib/torch/include/ATen/ops/_foreach_sin.h 2024-08-08T20:48:49.3948161Z inflating: build/lib/torch/include/ATen/ops/_foreach_sin_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3948593Z inflating: build/lib/torch/include/ATen/ops/_foreach_sin_cuda_dispatch.h 2024-08-08T20:48:49.3948947Z inflating: build/lib/torch/include/ATen/ops/_foreach_sin_native.h 2024-08-08T20:48:49.3949293Z inflating: build/lib/torch/include/ATen/ops/_foreach_sin_ops.h 2024-08-08T20:48:49.3949625Z inflating: build/lib/torch/include/ATen/ops/_foreach_sinh.h 2024-08-08T20:48:49.3950207Z inflating: build/lib/torch/include/ATen/ops/_foreach_sinh_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3950633Z inflating: build/lib/torch/include/ATen/ops/_foreach_sinh_cuda_dispatch.h 2024-08-08T20:48:49.3951015Z inflating: build/lib/torch/include/ATen/ops/_foreach_sinh_native.h 2024-08-08T20:48:49.3951361Z inflating: build/lib/torch/include/ATen/ops/_foreach_sinh_ops.h 2024-08-08T20:48:49.3951683Z inflating: build/lib/torch/include/ATen/ops/_foreach_sqrt.h 2024-08-08T20:48:49.3952257Z inflating: build/lib/torch/include/ATen/ops/_foreach_sqrt_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3952675Z inflating: build/lib/torch/include/ATen/ops/_foreach_sqrt_cuda_dispatch.h 2024-08-08T20:48:49.3953059Z inflating: build/lib/torch/include/ATen/ops/_foreach_sqrt_native.h 2024-08-08T20:48:49.3953416Z inflating: build/lib/torch/include/ATen/ops/_foreach_sqrt_ops.h 2024-08-08T20:48:49.3953726Z inflating: build/lib/torch/include/ATen/ops/_foreach_sub.h 2024-08-08T20:48:49.3954322Z inflating: build/lib/torch/include/ATen/ops/_foreach_sub_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3954737Z inflating: build/lib/torch/include/ATen/ops/_foreach_sub_cuda_dispatch.h 2024-08-08T20:48:49.3955157Z inflating: build/lib/torch/include/ATen/ops/_foreach_sub_native.h 2024-08-08T20:48:49.3955492Z inflating: build/lib/torch/include/ATen/ops/_foreach_sub_ops.h 2024-08-08T20:48:49.3955792Z inflating: build/lib/torch/include/ATen/ops/_foreach_tan.h 2024-08-08T20:48:49.3956378Z inflating: build/lib/torch/include/ATen/ops/_foreach_tan_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3956792Z inflating: build/lib/torch/include/ATen/ops/_foreach_tan_cuda_dispatch.h 2024-08-08T20:48:49.3957147Z inflating: build/lib/torch/include/ATen/ops/_foreach_tan_native.h 2024-08-08T20:48:49.3957598Z inflating: build/lib/torch/include/ATen/ops/_foreach_tan_ops.h 2024-08-08T20:48:49.3957921Z inflating: build/lib/torch/include/ATen/ops/_foreach_tanh.h 2024-08-08T20:48:49.3958744Z inflating: build/lib/torch/include/ATen/ops/_foreach_tanh_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3959192Z inflating: build/lib/torch/include/ATen/ops/_foreach_tanh_cuda_dispatch.h 2024-08-08T20:48:49.3959555Z inflating: build/lib/torch/include/ATen/ops/_foreach_tanh_native.h 2024-08-08T20:48:49.3959920Z inflating: build/lib/torch/include/ATen/ops/_foreach_tanh_ops.h 2024-08-08T20:48:49.3960241Z inflating: build/lib/torch/include/ATen/ops/_foreach_trunc.h 2024-08-08T20:48:49.3960825Z inflating: build/lib/torch/include/ATen/ops/_foreach_trunc_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3961269Z inflating: build/lib/torch/include/ATen/ops/_foreach_trunc_cuda_dispatch.h 2024-08-08T20:48:49.3961654Z inflating: build/lib/torch/include/ATen/ops/_foreach_trunc_native.h 2024-08-08T20:48:49.3962009Z inflating: build/lib/torch/include/ATen/ops/_foreach_trunc_ops.h 2024-08-08T20:48:49.3962455Z inflating: build/lib/torch/include/ATen/ops/_foreach_zero.h 2024-08-08T20:48:49.3963064Z inflating: build/lib/torch/include/ATen/ops/_foreach_zero_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3963505Z inflating: build/lib/torch/include/ATen/ops/_foreach_zero_cuda_dispatch.h 2024-08-08T20:48:49.3963872Z inflating: build/lib/torch/include/ATen/ops/_foreach_zero_native.h 2024-08-08T20:48:49.3964215Z inflating: build/lib/torch/include/ATen/ops/_foreach_zero_ops.h 2024-08-08T20:48:49.3964632Z inflating: build/lib/torch/include/ATen/ops/_functional_assert_async.h 2024-08-08T20:48:49.3965128Z inflating: build/lib/torch/include/ATen/ops/_functional_assert_async_cpu_dispatch.h 2024-08-08T20:48:49.3965583Z inflating: build/lib/torch/include/ATen/ops/_functional_assert_async_native.h 2024-08-08T20:48:49.3966032Z inflating: build/lib/torch/include/ATen/ops/_functional_assert_async_ops.h 2024-08-08T20:48:49.3966447Z inflating: build/lib/torch/include/ATen/ops/_functional_assert_scalar.h 2024-08-08T20:48:49.3967131Z inflating: build/lib/torch/include/ATen/ops/_functional_assert_scalar_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3967594Z inflating: build/lib/torch/include/ATen/ops/_functional_assert_scalar_native.h 2024-08-08T20:48:49.3968029Z inflating: build/lib/torch/include/ATen/ops/_functional_assert_scalar_ops.h 2024-08-08T20:48:49.3968500Z inflating: build/lib/torch/include/ATen/ops/_functional_sym_constrain_range.h 2024-08-08T20:48:49.3969203Z inflating: build/lib/torch/include/ATen/ops/_functional_sym_constrain_range_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3969712Z inflating: build/lib/torch/include/ATen/ops/_functional_sym_constrain_range_for_size.h 2024-08-08T20:48:49.3970486Z inflating: build/lib/torch/include/ATen/ops/_functional_sym_constrain_range_for_size_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3971053Z inflating: build/lib/torch/include/ATen/ops/_functional_sym_constrain_range_for_size_native.h 2024-08-08T20:48:49.3971615Z inflating: build/lib/torch/include/ATen/ops/_functional_sym_constrain_range_for_size_ops.h 2024-08-08T20:48:49.3972120Z inflating: build/lib/torch/include/ATen/ops/_functional_sym_constrain_range_native.h 2024-08-08T20:48:49.3972596Z inflating: build/lib/torch/include/ATen/ops/_functional_sym_constrain_range_ops.h 2024-08-08T20:48:49.3972938Z inflating: build/lib/torch/include/ATen/ops/_fused_adagrad.h 2024-08-08T20:48:49.3973527Z inflating: build/lib/torch/include/ATen/ops/_fused_adagrad_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3973947Z inflating: build/lib/torch/include/ATen/ops/_fused_adagrad_cpu_dispatch.h 2024-08-08T20:48:49.3974338Z inflating: build/lib/torch/include/ATen/ops/_fused_adagrad_native.h 2024-08-08T20:48:49.3974834Z inflating: build/lib/torch/include/ATen/ops/_fused_adagrad_ops.h 2024-08-08T20:48:49.3975146Z inflating: build/lib/torch/include/ATen/ops/_fused_adam.h 2024-08-08T20:48:49.3975727Z inflating: build/lib/torch/include/ATen/ops/_fused_adam_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3976133Z inflating: build/lib/torch/include/ATen/ops/_fused_adam_cpu_dispatch.h 2024-08-08T20:48:49.3976550Z inflating: build/lib/torch/include/ATen/ops/_fused_adam_cuda_dispatch.h 2024-08-08T20:48:49.3976901Z inflating: build/lib/torch/include/ATen/ops/_fused_adam_native.h 2024-08-08T20:48:49.3977231Z inflating: build/lib/torch/include/ATen/ops/_fused_adam_ops.h 2024-08-08T20:48:49.3977556Z inflating: build/lib/torch/include/ATen/ops/_fused_adamw.h 2024-08-08T20:48:49.3978130Z inflating: build/lib/torch/include/ATen/ops/_fused_adamw_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3978538Z inflating: build/lib/torch/include/ATen/ops/_fused_adamw_cpu_dispatch.h 2024-08-08T20:48:49.3978980Z inflating: build/lib/torch/include/ATen/ops/_fused_adamw_cuda_dispatch.h 2024-08-08T20:48:49.3979399Z inflating: build/lib/torch/include/ATen/ops/_fused_adamw_native.h 2024-08-08T20:48:49.3979832Z inflating: build/lib/torch/include/ATen/ops/_fused_adamw_ops.h 2024-08-08T20:48:49.3980171Z inflating: build/lib/torch/include/ATen/ops/_fused_dropout.h 2024-08-08T20:48:49.3980776Z inflating: build/lib/torch/include/ATen/ops/_fused_dropout_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3981207Z inflating: build/lib/torch/include/ATen/ops/_fused_dropout_cuda_dispatch.h 2024-08-08T20:48:49.3981580Z inflating: build/lib/torch/include/ATen/ops/_fused_dropout_native.h 2024-08-08T20:48:49.3981953Z inflating: build/lib/torch/include/ATen/ops/_fused_dropout_ops.h 2024-08-08T20:48:49.3982405Z inflating: build/lib/torch/include/ATen/ops/_fused_moving_avg_obs_fq_helper.h 2024-08-08T20:48:49.3983110Z inflating: build/lib/torch/include/ATen/ops/_fused_moving_avg_obs_fq_helper_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3983678Z inflating: build/lib/torch/include/ATen/ops/_fused_moving_avg_obs_fq_helper_cpu_dispatch.h 2024-08-08T20:48:49.3984230Z inflating: build/lib/torch/include/ATen/ops/_fused_moving_avg_obs_fq_helper_cuda_dispatch.h 2024-08-08T20:48:49.3984743Z inflating: build/lib/torch/include/ATen/ops/_fused_moving_avg_obs_fq_helper_native.h 2024-08-08T20:48:49.3985221Z inflating: build/lib/torch/include/ATen/ops/_fused_moving_avg_obs_fq_helper_ops.h 2024-08-08T20:48:49.3985570Z inflating: build/lib/torch/include/ATen/ops/_fused_sdp_choice.h 2024-08-08T20:48:49.3986032Z inflating: build/lib/torch/include/ATen/ops/_fused_sdp_choice_cpu_dispatch.h 2024-08-08T20:48:49.3986484Z inflating: build/lib/torch/include/ATen/ops/_fused_sdp_choice_cuda_dispatch.h 2024-08-08T20:48:49.3986933Z inflating: build/lib/torch/include/ATen/ops/_fused_sdp_choice_meta_dispatch.h 2024-08-08T20:48:49.3987345Z inflating: build/lib/torch/include/ATen/ops/_fused_sdp_choice_native.h 2024-08-08T20:48:49.3987729Z inflating: build/lib/torch/include/ATen/ops/_fused_sdp_choice_ops.h 2024-08-08T20:48:49.3988043Z inflating: build/lib/torch/include/ATen/ops/_fused_sgd.h 2024-08-08T20:48:49.3988611Z inflating: build/lib/torch/include/ATen/ops/_fused_sgd_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3989003Z inflating: build/lib/torch/include/ATen/ops/_fused_sgd_cpu_dispatch.h 2024-08-08T20:48:49.3989421Z inflating: build/lib/torch/include/ATen/ops/_fused_sgd_cuda_dispatch.h 2024-08-08T20:48:49.3989807Z inflating: build/lib/torch/include/ATen/ops/_fused_sgd_native.h 2024-08-08T20:48:49.3990132Z inflating: build/lib/torch/include/ATen/ops/_fused_sgd_ops.h 2024-08-08T20:48:49.3990430Z inflating: build/lib/torch/include/ATen/ops/_fw_primal.h 2024-08-08T20:48:49.3990992Z inflating: build/lib/torch/include/ATen/ops/_fw_primal_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3991320Z inflating: build/lib/torch/include/ATen/ops/_fw_primal_copy.h 2024-08-08T20:48:49.3992036Z inflating: build/lib/torch/include/ATen/ops/_fw_primal_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.3992746Z inflating: build/lib/torch/include/ATen/ops/_fw_primal_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.3993148Z inflating: build/lib/torch/include/ATen/ops/_fw_primal_copy_native.h 2024-08-08T20:48:49.3993505Z inflating: build/lib/torch/include/ATen/ops/_fw_primal_copy_ops.h 2024-08-08T20:48:49.3993851Z inflating: build/lib/torch/include/ATen/ops/_fw_primal_native.h 2024-08-08T20:48:49.3994189Z inflating: build/lib/torch/include/ATen/ops/_fw_primal_ops.h 2024-08-08T20:48:49.3994582Z inflating: build/lib/torch/include/ATen/ops/_gather_sparse_backward.h 2024-08-08T20:48:49.3995235Z inflating: build/lib/torch/include/ATen/ops/_gather_sparse_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3995699Z inflating: build/lib/torch/include/ATen/ops/_gather_sparse_backward_native.h 2024-08-08T20:48:49.3996131Z inflating: build/lib/torch/include/ATen/ops/_gather_sparse_backward_ops.h 2024-08-08T20:48:49.3996668Z inflating: build/lib/torch/include/ATen/ops/_grid_sampler_2d_cpu_fallback.h 2024-08-08T20:48:49.3997178Z inflating: build/lib/torch/include/ATen/ops/_grid_sampler_2d_cpu_fallback_backward.h 2024-08-08T20:48:49.3997926Z inflating: build/lib/torch/include/ATen/ops/_grid_sampler_2d_cpu_fallback_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.3998488Z inflating: build/lib/torch/include/ATen/ops/_grid_sampler_2d_cpu_fallback_backward_native.h 2024-08-08T20:48:49.3999013Z inflating: build/lib/torch/include/ATen/ops/_grid_sampler_2d_cpu_fallback_backward_ops.h 2024-08-08T20:48:49.3999704Z inflating: build/lib/torch/include/ATen/ops/_grid_sampler_2d_cpu_fallback_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4000199Z inflating: build/lib/torch/include/ATen/ops/_grid_sampler_2d_cpu_fallback_native.h 2024-08-08T20:48:49.4000665Z inflating: build/lib/torch/include/ATen/ops/_grid_sampler_2d_cpu_fallback_ops.h 2024-08-08T20:48:49.4001147Z inflating: build/lib/torch/include/ATen/ops/_has_compatible_shallow_copy_type.h 2024-08-08T20:48:49.4001872Z inflating: build/lib/torch/include/ATen/ops/_has_compatible_shallow_copy_type_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4002387Z inflating: build/lib/torch/include/ATen/ops/_has_compatible_shallow_copy_type_native.h 2024-08-08T20:48:49.4002891Z inflating: build/lib/torch/include/ATen/ops/_has_compatible_shallow_copy_type_ops.h 2024-08-08T20:48:49.4003282Z inflating: build/lib/torch/include/ATen/ops/_has_same_storage_numel.h 2024-08-08T20:48:49.4003927Z inflating: build/lib/torch/include/ATen/ops/_has_same_storage_numel_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4004387Z inflating: build/lib/torch/include/ATen/ops/_has_same_storage_numel_native.h 2024-08-08T20:48:49.4004804Z inflating: build/lib/torch/include/ATen/ops/_has_same_storage_numel_ops.h 2024-08-08T20:48:49.4005210Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_bin_edges.h 2024-08-08T20:48:49.4005867Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_bin_edges_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4006346Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_bin_edges_cpu_dispatch.h 2024-08-08T20:48:49.4006796Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_bin_edges_native.h 2024-08-08T20:48:49.4007215Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_bin_edges_ops.h 2024-08-08T20:48:49.4007624Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_cts.h 2024-08-08T20:48:49.4008306Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_cts_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4008811Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_cts_cpu_dispatch.h 2024-08-08T20:48:49.4009377Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_cts_native.h 2024-08-08T20:48:49.4009810Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_cts_ops.h 2024-08-08T20:48:49.4010248Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_tensors.h 2024-08-08T20:48:49.4010958Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_tensors_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4011485Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_tensors_cpu_dispatch.h 2024-08-08T20:48:49.4011974Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_tensors_native.h 2024-08-08T20:48:49.4012452Z inflating: build/lib/torch/include/ATen/ops/_histogramdd_from_bin_tensors_ops.h 2024-08-08T20:48:49.4012783Z inflating: build/lib/torch/include/ATen/ops/_index_put_impl.h 2024-08-08T20:48:49.4013394Z inflating: build/lib/torch/include/ATen/ops/_index_put_impl_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4013835Z inflating: build/lib/torch/include/ATen/ops/_index_put_impl_cpu_dispatch.h 2024-08-08T20:48:49.4014271Z inflating: build/lib/torch/include/ATen/ops/_index_put_impl_cuda_dispatch.h 2024-08-08T20:48:49.4014787Z inflating: build/lib/torch/include/ATen/ops/_index_put_impl_meta_dispatch.h 2024-08-08T20:48:49.4015190Z inflating: build/lib/torch/include/ATen/ops/_index_put_impl_native.h 2024-08-08T20:48:49.4015549Z inflating: build/lib/torch/include/ATen/ops/_index_put_impl_ops.h 2024-08-08T20:48:49.4015844Z inflating: build/lib/torch/include/ATen/ops/_indices.h 2024-08-08T20:48:49.4016162Z inflating: build/lib/torch/include/ATen/ops/_indices_copy.h 2024-08-08T20:48:49.4016756Z inflating: build/lib/torch/include/ATen/ops/_indices_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4017454Z inflating: build/lib/torch/include/ATen/ops/_indices_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4017823Z inflating: build/lib/torch/include/ATen/ops/_indices_copy_native.h 2024-08-08T20:48:49.4018191Z inflating: build/lib/torch/include/ATen/ops/_indices_copy_ops.h 2024-08-08T20:48:49.4018527Z inflating: build/lib/torch/include/ATen/ops/_indices_native.h 2024-08-08T20:48:49.4018832Z inflating: build/lib/torch/include/ATen/ops/_indices_ops.h 2024-08-08T20:48:49.4019116Z inflating: build/lib/torch/include/ATen/ops/_int_mm.h 2024-08-08T20:48:49.4019486Z inflating: build/lib/torch/include/ATen/ops/_int_mm_cpu_dispatch.h 2024-08-08T20:48:49.4019862Z inflating: build/lib/torch/include/ATen/ops/_int_mm_cuda_dispatch.h 2024-08-08T20:48:49.4020203Z inflating: build/lib/torch/include/ATen/ops/_int_mm_native.h 2024-08-08T20:48:49.4020497Z inflating: build/lib/torch/include/ATen/ops/_int_mm_ops.h 2024-08-08T20:48:49.4020811Z inflating: build/lib/torch/include/ATen/ops/_is_all_true.h 2024-08-08T20:48:49.4021385Z inflating: build/lib/torch/include/ATen/ops/_is_all_true_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4021748Z inflating: build/lib/torch/include/ATen/ops/_is_all_true_native.h 2024-08-08T20:48:49.4022097Z inflating: build/lib/torch/include/ATen/ops/_is_all_true_ops.h 2024-08-08T20:48:49.4022405Z inflating: build/lib/torch/include/ATen/ops/_is_any_true.h 2024-08-08T20:48:49.4022976Z inflating: build/lib/torch/include/ATen/ops/_is_any_true_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4023346Z inflating: build/lib/torch/include/ATen/ops/_is_any_true_native.h 2024-08-08T20:48:49.4023679Z inflating: build/lib/torch/include/ATen/ops/_is_any_true_ops.h 2024-08-08T20:48:49.4024020Z inflating: build/lib/torch/include/ATen/ops/_is_zerotensor.h 2024-08-08T20:48:49.4024611Z inflating: build/lib/torch/include/ATen/ops/_is_zerotensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4024990Z inflating: build/lib/torch/include/ATen/ops/_is_zerotensor_native.h 2024-08-08T20:48:49.4025356Z inflating: build/lib/torch/include/ATen/ops/_is_zerotensor_ops.h 2024-08-08T20:48:49.4025945Z inflating: build/lib/torch/include/ATen/ops/_jagged_to_padded_dense_forward.h 2024-08-08T20:48:49.4026495Z inflating: build/lib/torch/include/ATen/ops/_jagged_to_padded_dense_forward_cpu_dispatch.h 2024-08-08T20:48:49.4027058Z inflating: build/lib/torch/include/ATen/ops/_jagged_to_padded_dense_forward_cuda_dispatch.h 2024-08-08T20:48:49.4027555Z inflating: build/lib/torch/include/ATen/ops/_jagged_to_padded_dense_forward_native.h 2024-08-08T20:48:49.4028046Z inflating: build/lib/torch/include/ATen/ops/_jagged_to_padded_dense_forward_ops.h 2024-08-08T20:48:49.4028343Z inflating: build/lib/torch/include/ATen/ops/_lazy_clone.h 2024-08-08T20:48:49.4028910Z inflating: build/lib/torch/include/ATen/ops/_lazy_clone_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4029279Z inflating: build/lib/torch/include/ATen/ops/_lazy_clone_native.h 2024-08-08T20:48:49.4029609Z inflating: build/lib/torch/include/ATen/ops/_lazy_clone_ops.h 2024-08-08T20:48:49.4029985Z inflating: build/lib/torch/include/ATen/ops/_linalg_check_errors.h 2024-08-08T20:48:49.4030705Z inflating: build/lib/torch/include/ATen/ops/_linalg_check_errors_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4031138Z inflating: build/lib/torch/include/ATen/ops/_linalg_check_errors_native.h 2024-08-08T20:48:49.4031539Z inflating: build/lib/torch/include/ATen/ops/_linalg_check_errors_ops.h 2024-08-08T20:48:49.4031848Z inflating: build/lib/torch/include/ATen/ops/_linalg_det.h 2024-08-08T20:48:49.4032528Z inflating: build/lib/torch/include/ATen/ops/_linalg_det_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4032941Z inflating: build/lib/torch/include/ATen/ops/_linalg_det_cpu_dispatch.h 2024-08-08T20:48:49.4033357Z inflating: build/lib/torch/include/ATen/ops/_linalg_det_cuda_dispatch.h 2024-08-08T20:48:49.4033693Z inflating: build/lib/torch/include/ATen/ops/_linalg_det_meta.h 2024-08-08T20:48:49.4034110Z inflating: build/lib/torch/include/ATen/ops/_linalg_det_meta_dispatch.h 2024-08-08T20:48:49.4034468Z inflating: build/lib/torch/include/ATen/ops/_linalg_det_native.h 2024-08-08T20:48:49.4034806Z inflating: build/lib/torch/include/ATen/ops/_linalg_det_ops.h 2024-08-08T20:48:49.4035165Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigh.h 2024-08-08T20:48:49.4035855Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigh_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4036320Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigh_cpu_dispatch.h 2024-08-08T20:48:49.4036741Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigh_cuda_dispatch.h 2024-08-08T20:48:49.4037087Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigh_meta.h 2024-08-08T20:48:49.4037521Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigh_meta_dispatch.h 2024-08-08T20:48:49.4037876Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigh_native.h 2024-08-08T20:48:49.4038242Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigh_ops.h 2024-08-08T20:48:49.4038573Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigvals.h 2024-08-08T20:48:49.4039007Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigvals_cpu_dispatch.h 2024-08-08T20:48:49.4039458Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigvals_cuda_dispatch.h 2024-08-08T20:48:49.4039842Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigvals_native.h 2024-08-08T20:48:49.4040195Z inflating: build/lib/torch/include/ATen/ops/_linalg_eigvals_ops.h 2024-08-08T20:48:49.4040540Z inflating: build/lib/torch/include/ATen/ops/_linalg_slogdet.h 2024-08-08T20:48:49.4041249Z inflating: build/lib/torch/include/ATen/ops/_linalg_slogdet_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4041681Z inflating: build/lib/torch/include/ATen/ops/_linalg_slogdet_cpu_dispatch.h 2024-08-08T20:48:49.4042131Z inflating: build/lib/torch/include/ATen/ops/_linalg_slogdet_cuda_dispatch.h 2024-08-08T20:48:49.4042597Z inflating: build/lib/torch/include/ATen/ops/_linalg_slogdet_meta.h 2024-08-08T20:48:49.4043044Z inflating: build/lib/torch/include/ATen/ops/_linalg_slogdet_meta_dispatch.h 2024-08-08T20:48:49.4043433Z inflating: build/lib/torch/include/ATen/ops/_linalg_slogdet_native.h 2024-08-08T20:48:49.4043795Z inflating: build/lib/torch/include/ATen/ops/_linalg_slogdet_ops.h 2024-08-08T20:48:49.4044141Z inflating: build/lib/torch/include/ATen/ops/_linalg_solve_ex.h 2024-08-08T20:48:49.4044851Z inflating: build/lib/torch/include/ATen/ops/_linalg_solve_ex_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4045279Z inflating: build/lib/torch/include/ATen/ops/_linalg_solve_ex_cpu_dispatch.h 2024-08-08T20:48:49.4045738Z inflating: build/lib/torch/include/ATen/ops/_linalg_solve_ex_cuda_dispatch.h 2024-08-08T20:48:49.4046112Z inflating: build/lib/torch/include/ATen/ops/_linalg_solve_ex_meta.h 2024-08-08T20:48:49.4046569Z inflating: build/lib/torch/include/ATen/ops/_linalg_solve_ex_meta_dispatch.h 2024-08-08T20:48:49.4046969Z inflating: build/lib/torch/include/ATen/ops/_linalg_solve_ex_native.h 2024-08-08T20:48:49.4047414Z inflating: build/lib/torch/include/ATen/ops/_linalg_solve_ex_ops.h 2024-08-08T20:48:49.4047741Z inflating: build/lib/torch/include/ATen/ops/_linalg_svd.h 2024-08-08T20:48:49.4048417Z inflating: build/lib/torch/include/ATen/ops/_linalg_svd_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4048815Z inflating: build/lib/torch/include/ATen/ops/_linalg_svd_cpu_dispatch.h 2024-08-08T20:48:49.4049239Z inflating: build/lib/torch/include/ATen/ops/_linalg_svd_cuda_dispatch.h 2024-08-08T20:48:49.4049579Z inflating: build/lib/torch/include/ATen/ops/_linalg_svd_meta.h 2024-08-08T20:48:49.4050008Z inflating: build/lib/torch/include/ATen/ops/_linalg_svd_meta_dispatch.h 2024-08-08T20:48:49.4050368Z inflating: build/lib/torch/include/ATen/ops/_linalg_svd_native.h 2024-08-08T20:48:49.4050707Z inflating: build/lib/torch/include/ATen/ops/_linalg_svd_ops.h 2024-08-08T20:48:49.4051080Z inflating: build/lib/torch/include/ATen/ops/_local_scalar_dense.h 2024-08-08T20:48:49.4051547Z inflating: build/lib/torch/include/ATen/ops/_local_scalar_dense_cpu_dispatch.h 2024-08-08T20:48:49.4052016Z inflating: build/lib/torch/include/ATen/ops/_local_scalar_dense_cuda_dispatch.h 2024-08-08T20:48:49.4052451Z inflating: build/lib/torch/include/ATen/ops/_local_scalar_dense_native.h 2024-08-08T20:48:49.4052844Z inflating: build/lib/torch/include/ATen/ops/_local_scalar_dense_ops.h 2024-08-08T20:48:49.4053148Z inflating: build/lib/torch/include/ATen/ops/_log_softmax.h 2024-08-08T20:48:49.4053577Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_backward_data.h 2024-08-08T20:48:49.4054350Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_backward_data_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4054869Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_backward_data_cpu_dispatch.h 2024-08-08T20:48:49.4055390Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_backward_data_cuda_dispatch.h 2024-08-08T20:48:49.4055845Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_backward_data_meta.h 2024-08-08T20:48:49.4056378Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_backward_data_meta_dispatch.h 2024-08-08T20:48:49.4056840Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_backward_data_native.h 2024-08-08T20:48:49.4057285Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_backward_data_ops.h 2024-08-08T20:48:49.4057985Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4058620Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_cpu_dispatch.h 2024-08-08T20:48:49.4059061Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_cuda_dispatch.h 2024-08-08T20:48:49.4059410Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_meta.h 2024-08-08T20:48:49.4059981Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_meta_dispatch.h 2024-08-08T20:48:49.4060355Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_native.h 2024-08-08T20:48:49.4060720Z inflating: build/lib/torch/include/ATen/ops/_log_softmax_ops.h 2024-08-08T20:48:49.4061054Z inflating: build/lib/torch/include/ATen/ops/_logcumsumexp.h 2024-08-08T20:48:49.4061468Z inflating: build/lib/torch/include/ATen/ops/_logcumsumexp_cpu_dispatch.h 2024-08-08T20:48:49.4061890Z inflating: build/lib/torch/include/ATen/ops/_logcumsumexp_cuda_dispatch.h 2024-08-08T20:48:49.4062279Z inflating: build/lib/torch/include/ATen/ops/_logcumsumexp_native.h 2024-08-08T20:48:49.4062629Z inflating: build/lib/torch/include/ATen/ops/_logcumsumexp_ops.h 2024-08-08T20:48:49.4062915Z inflating: build/lib/torch/include/ATen/ops/_lstm_mps.h 2024-08-08T20:48:49.4063490Z inflating: build/lib/torch/include/ATen/ops/_lstm_mps_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4063840Z inflating: build/lib/torch/include/ATen/ops/_lstm_mps_native.h 2024-08-08T20:48:49.4064255Z inflating: build/lib/torch/include/ATen/ops/_lstm_mps_ops.h 2024-08-08T20:48:49.4064607Z inflating: build/lib/torch/include/ATen/ops/_lu_with_info.h 2024-08-08T20:48:49.4065186Z inflating: build/lib/torch/include/ATen/ops/_lu_with_info_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4065564Z inflating: build/lib/torch/include/ATen/ops/_lu_with_info_native.h 2024-08-08T20:48:49.4065910Z inflating: build/lib/torch/include/ATen/ops/_lu_with_info_ops.h 2024-08-08T20:48:49.4066238Z inflating: build/lib/torch/include/ATen/ops/_make_dep_token.h 2024-08-08T20:48:49.4066682Z inflating: build/lib/torch/include/ATen/ops/_make_dep_token_cpu_dispatch.h 2024-08-08T20:48:49.4067067Z inflating: build/lib/torch/include/ATen/ops/_make_dep_token_native.h 2024-08-08T20:48:49.4067423Z inflating: build/lib/torch/include/ATen/ops/_make_dep_token_ops.h 2024-08-08T20:48:49.4067736Z inflating: build/lib/torch/include/ATen/ops/_make_dual.h 2024-08-08T20:48:49.4068308Z inflating: build/lib/torch/include/ATen/ops/_make_dual_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4068639Z inflating: build/lib/torch/include/ATen/ops/_make_dual_copy.h 2024-08-08T20:48:49.4069247Z inflating: build/lib/torch/include/ATen/ops/_make_dual_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4069950Z inflating: build/lib/torch/include/ATen/ops/_make_dual_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4070351Z inflating: build/lib/torch/include/ATen/ops/_make_dual_copy_native.h 2024-08-08T20:48:49.4070711Z inflating: build/lib/torch/include/ATen/ops/_make_dual_copy_ops.h 2024-08-08T20:48:49.4071054Z inflating: build/lib/torch/include/ATen/ops/_make_dual_native.h 2024-08-08T20:48:49.4071409Z inflating: build/lib/torch/include/ATen/ops/_make_dual_ops.h 2024-08-08T20:48:49.4071890Z inflating: build/lib/torch/include/ATen/ops/_make_per_channel_quantized_tensor.h 2024-08-08T20:48:49.4072639Z inflating: build/lib/torch/include/ATen/ops/_make_per_channel_quantized_tensor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4073203Z inflating: build/lib/torch/include/ATen/ops/_make_per_channel_quantized_tensor_cpu_dispatch.h 2024-08-08T20:48:49.4073767Z inflating: build/lib/torch/include/ATen/ops/_make_per_channel_quantized_tensor_cuda_dispatch.h 2024-08-08T20:48:49.4074302Z inflating: build/lib/torch/include/ATen/ops/_make_per_channel_quantized_tensor_native.h 2024-08-08T20:48:49.4074803Z inflating: build/lib/torch/include/ATen/ops/_make_per_channel_quantized_tensor_ops.h 2024-08-08T20:48:49.4075267Z inflating: build/lib/torch/include/ATen/ops/_make_per_tensor_quantized_tensor.h 2024-08-08T20:48:49.4075999Z inflating: build/lib/torch/include/ATen/ops/_make_per_tensor_quantized_tensor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4076651Z inflating: build/lib/torch/include/ATen/ops/_make_per_tensor_quantized_tensor_cpu_dispatch.h 2024-08-08T20:48:49.4077239Z inflating: build/lib/torch/include/ATen/ops/_make_per_tensor_quantized_tensor_cuda_dispatch.h 2024-08-08T20:48:49.4077750Z inflating: build/lib/torch/include/ATen/ops/_make_per_tensor_quantized_tensor_native.h 2024-08-08T20:48:49.4078244Z inflating: build/lib/torch/include/ATen/ops/_make_per_tensor_quantized_tensor_ops.h 2024-08-08T20:48:49.4078578Z inflating: build/lib/torch/include/ATen/ops/_masked_scale.h 2024-08-08T20:48:49.4079164Z inflating: build/lib/torch/include/ATen/ops/_masked_scale_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4079586Z inflating: build/lib/torch/include/ATen/ops/_masked_scale_cuda_dispatch.h 2024-08-08T20:48:49.4079980Z inflating: build/lib/torch/include/ATen/ops/_masked_scale_native.h 2024-08-08T20:48:49.4080327Z inflating: build/lib/torch/include/ATen/ops/_masked_scale_ops.h 2024-08-08T20:48:49.4080677Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax.h 2024-08-08T20:48:49.4081086Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_backward.h 2024-08-08T20:48:49.4082279Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4082959Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_backward_cpu_dispatch.h 2024-08-08T20:48:49.4083837Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_backward_cuda_dispatch.h 2024-08-08T20:48:49.4084743Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_backward_native.h 2024-08-08T20:48:49.4085758Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_backward_ops.h 2024-08-08T20:48:49.4086915Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4087726Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_cpu_dispatch.h 2024-08-08T20:48:49.4088703Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_cuda_dispatch.h 2024-08-08T20:48:49.4089655Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_native.h 2024-08-08T20:48:49.4090643Z inflating: build/lib/torch/include/ATen/ops/_masked_softmax_ops.h 2024-08-08T20:48:49.4091601Z inflating: build/lib/torch/include/ATen/ops/_mixed_dtypes_linear.h 2024-08-08T20:48:49.4092644Z inflating: build/lib/torch/include/ATen/ops/_mixed_dtypes_linear_cuda_dispatch.h 2024-08-08T20:48:49.4093505Z inflating: build/lib/torch/include/ATen/ops/_mixed_dtypes_linear_native.h 2024-08-08T20:48:49.4094519Z inflating: build/lib/torch/include/ATen/ops/_mixed_dtypes_linear_ops.h 2024-08-08T20:48:49.4095400Z inflating: build/lib/torch/include/ATen/ops/_mkldnn_reshape.h 2024-08-08T20:48:49.4096664Z inflating: build/lib/torch/include/ATen/ops/_mkldnn_reshape_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4097589Z inflating: build/lib/torch/include/ATen/ops/_mkldnn_reshape_native.h 2024-08-08T20:48:49.4099628Z inflating: build/lib/torch/include/ATen/ops/_mkldnn_reshape_ops.h 2024-08-08T20:48:49.4100265Z inflating: build/lib/torch/include/ATen/ops/_mkldnn_transpose.h 2024-08-08T20:48:49.4101459Z inflating: build/lib/torch/include/ATen/ops/_mkldnn_transpose_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4102236Z inflating: build/lib/torch/include/ATen/ops/_mkldnn_transpose_meta_dispatch.h 2024-08-08T20:48:49.4103107Z inflating: build/lib/torch/include/ATen/ops/_mkldnn_transpose_native.h 2024-08-08T20:48:49.4104188Z inflating: build/lib/torch/include/ATen/ops/_mkldnn_transpose_ops.h 2024-08-08T20:48:49.4105681Z inflating: build/lib/torch/include/ATen/ops/_mps_convolution.h 2024-08-08T20:48:49.4106610Z inflating: build/lib/torch/include/ATen/ops/_mps_convolution_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4107468Z inflating: build/lib/torch/include/ATen/ops/_mps_convolution_native.h 2024-08-08T20:48:49.4108484Z inflating: build/lib/torch/include/ATen/ops/_mps_convolution_ops.h 2024-08-08T20:48:49.4109873Z inflating: build/lib/torch/include/ATen/ops/_mps_convolution_transpose.h 2024-08-08T20:48:49.4111185Z inflating: build/lib/torch/include/ATen/ops/_mps_convolution_transpose_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4111885Z inflating: build/lib/torch/include/ATen/ops/_mps_convolution_transpose_native.h 2024-08-08T20:48:49.4112950Z inflating: build/lib/torch/include/ATen/ops/_mps_convolution_transpose_ops.h 2024-08-08T20:48:49.4114104Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit.h 2024-08-08T20:48:49.4115347Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4116241Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit_cpu_dispatch.h 2024-08-08T20:48:49.4117376Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit_cuda_dispatch.h 2024-08-08T20:48:49.4118510Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit_native.h 2024-08-08T20:48:49.4119568Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit_no_training.h 2024-08-08T20:48:49.4121009Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit_no_training_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4121762Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit_no_training_native.h 2024-08-08T20:48:49.4122772Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit_no_training_ops.h 2024-08-08T20:48:49.4124007Z inflating: build/lib/torch/include/ATen/ops/_native_batch_norm_legit_ops.h 2024-08-08T20:48:49.4125145Z inflating: build/lib/torch/include/ATen/ops/_native_multi_head_attention.h 2024-08-08T20:48:49.4126477Z inflating: build/lib/torch/include/ATen/ops/_native_multi_head_attention_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4127283Z inflating: build/lib/torch/include/ATen/ops/_native_multi_head_attention_cpu_dispatch.h 2024-08-08T20:48:49.4128354Z inflating: build/lib/torch/include/ATen/ops/_native_multi_head_attention_cuda_dispatch.h 2024-08-08T20:48:49.4129306Z inflating: build/lib/torch/include/ATen/ops/_native_multi_head_attention_native.h 2024-08-08T20:48:49.4130505Z inflating: build/lib/torch/include/ATen/ops/_native_multi_head_attention_ops.h 2024-08-08T20:48:49.4131236Z inflating: build/lib/torch/include/ATen/ops/_neg_view.h 2024-08-08T20:48:49.4133060Z inflating: build/lib/torch/include/ATen/ops/_neg_view_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4133406Z inflating: build/lib/torch/include/ATen/ops/_neg_view_copy.h 2024-08-08T20:48:49.4134579Z inflating: build/lib/torch/include/ATen/ops/_neg_view_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4135731Z inflating: build/lib/torch/include/ATen/ops/_neg_view_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4136347Z inflating: build/lib/torch/include/ATen/ops/_neg_view_copy_native.h 2024-08-08T20:48:49.4137353Z inflating: build/lib/torch/include/ATen/ops/_neg_view_copy_ops.h 2024-08-08T20:48:49.4138281Z inflating: build/lib/torch/include/ATen/ops/_neg_view_native.h 2024-08-08T20:48:49.4139195Z inflating: build/lib/torch/include/ATen/ops/_neg_view_ops.h 2024-08-08T20:48:49.4140351Z inflating: build/lib/torch/include/ATen/ops/_nested_compute_contiguous_strides_offsets.h 2024-08-08T20:48:49.4141475Z inflating: build/lib/torch/include/ATen/ops/_nested_compute_contiguous_strides_offsets_cpu_dispatch.h 2024-08-08T20:48:49.4142464Z inflating: build/lib/torch/include/ATen/ops/_nested_compute_contiguous_strides_offsets_cuda_dispatch.h 2024-08-08T20:48:49.4143399Z inflating: build/lib/torch/include/ATen/ops/_nested_compute_contiguous_strides_offsets_native.h 2024-08-08T20:48:49.4144364Z inflating: build/lib/torch/include/ATen/ops/_nested_compute_contiguous_strides_offsets_ops.h 2024-08-08T20:48:49.4145323Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded.h 2024-08-08T20:48:49.4146345Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded_and_nested_example.h 2024-08-08T20:48:49.4147590Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded_and_nested_example_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4148341Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded_and_nested_example_native.h 2024-08-08T20:48:49.4149337Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded_and_nested_example_ops.h 2024-08-08T20:48:49.4150441Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4151209Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded_cpu_dispatch.h 2024-08-08T20:48:49.4152159Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded_cuda_dispatch.h 2024-08-08T20:48:49.4153125Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded_native.h 2024-08-08T20:48:49.4154158Z inflating: build/lib/torch/include/ATen/ops/_nested_from_padded_ops.h 2024-08-08T20:48:49.4155183Z inflating: build/lib/torch/include/ATen/ops/_nested_get_jagged_dummy.h 2024-08-08T20:48:49.4156098Z inflating: build/lib/torch/include/ATen/ops/_nested_get_jagged_dummy_native.h 2024-08-08T20:48:49.4157130Z inflating: build/lib/torch/include/ATen/ops/_nested_get_jagged_dummy_ops.h 2024-08-08T20:48:49.4157962Z inflating: build/lib/torch/include/ATen/ops/_nested_get_lengths.h 2024-08-08T20:48:49.4159072Z inflating: build/lib/torch/include/ATen/ops/_nested_get_lengths_native.h 2024-08-08T20:48:49.4160223Z inflating: build/lib/torch/include/ATen/ops/_nested_get_lengths_ops.h 2024-08-08T20:48:49.4161208Z inflating: build/lib/torch/include/ATen/ops/_nested_get_max_seqlen.h 2024-08-08T20:48:49.4162127Z inflating: build/lib/torch/include/ATen/ops/_nested_get_max_seqlen_native.h 2024-08-08T20:48:49.4163085Z inflating: build/lib/torch/include/ATen/ops/_nested_get_max_seqlen_ops.h 2024-08-08T20:48:49.4164184Z inflating: build/lib/torch/include/ATen/ops/_nested_get_min_seqlen.h 2024-08-08T20:48:49.4165894Z inflating: build/lib/torch/include/ATen/ops/_nested_get_min_seqlen_native.h 2024-08-08T20:48:49.4166620Z inflating: build/lib/torch/include/ATen/ops/_nested_get_min_seqlen_ops.h 2024-08-08T20:48:49.4167521Z inflating: build/lib/torch/include/ATen/ops/_nested_get_offsets.h 2024-08-08T20:48:49.4168531Z inflating: build/lib/torch/include/ATen/ops/_nested_get_offsets_native.h 2024-08-08T20:48:49.4169499Z inflating: build/lib/torch/include/ATen/ops/_nested_get_offsets_ops.h 2024-08-08T20:48:49.4170396Z inflating: build/lib/torch/include/ATen/ops/_nested_get_ragged_idx.h 2024-08-08T20:48:49.4171352Z inflating: build/lib/torch/include/ATen/ops/_nested_get_ragged_idx_native.h 2024-08-08T20:48:49.4172337Z inflating: build/lib/torch/include/ATen/ops/_nested_get_ragged_idx_ops.h 2024-08-08T20:48:49.4173190Z inflating: build/lib/torch/include/ATen/ops/_nested_get_values.h 2024-08-08T20:48:49.4174107Z inflating: build/lib/torch/include/ATen/ops/_nnpack_available.h 2024-08-08T20:48:49.4175430Z inflating: build/lib/torch/include/ATen/ops/_nested_get_values_copy.h 2024-08-08T20:48:49.4176639Z inflating: build/lib/torch/include/ATen/ops/_nested_get_values_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4177729Z inflating: build/lib/torch/include/ATen/ops/_nested_get_values_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4178386Z inflating: build/lib/torch/include/ATen/ops/_nested_get_values_copy_native.h 2024-08-08T20:48:49.4179336Z inflating: build/lib/torch/include/ATen/ops/_nested_get_values_copy_ops.h 2024-08-08T20:48:49.4180196Z inflating: build/lib/torch/include/ATen/ops/_nested_get_values_native.h 2024-08-08T20:48:49.4181198Z inflating: build/lib/torch/include/ATen/ops/_nested_get_values_ops.h 2024-08-08T20:48:49.4182277Z inflating: build/lib/torch/include/ATen/ops/_nested_select_backward.h 2024-08-08T20:48:49.4183155Z inflating: build/lib/torch/include/ATen/ops/_nested_select_backward_native.h 2024-08-08T20:48:49.4184145Z inflating: build/lib/torch/include/ATen/ops/_nested_select_backward_ops.h 2024-08-08T20:48:49.4185053Z inflating: build/lib/torch/include/ATen/ops/_nested_sum_backward.h 2024-08-08T20:48:49.4186010Z inflating: build/lib/torch/include/ATen/ops/_nested_sum_backward_native.h 2024-08-08T20:48:49.4186985Z inflating: build/lib/torch/include/ATen/ops/_nested_sum_backward_ops.h 2024-08-08T20:48:49.4187956Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask.h 2024-08-08T20:48:49.4189220Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4190030Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_cpu_dispatch.h 2024-08-08T20:48:49.4190997Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_cuda_dispatch.h 2024-08-08T20:48:49.4192014Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_left_aligned.h 2024-08-08T20:48:49.4193204Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_left_aligned_cpu_dispatch.h 2024-08-08T20:48:49.4194267Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_left_aligned_cuda_dispatch.h 2024-08-08T20:48:49.4195079Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_left_aligned_native.h 2024-08-08T20:48:49.4196244Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_left_aligned_ops.h 2024-08-08T20:48:49.4197113Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_native.h 2024-08-08T20:48:49.4198190Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_mask_ops.h 2024-08-08T20:48:49.4199263Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_tensor_list.h 2024-08-08T20:48:49.4200530Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_tensor_list_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4201274Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_tensor_list_native.h 2024-08-08T20:48:49.4202312Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_from_tensor_list_ops.h 2024-08-08T20:48:49.4203176Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_size.h 2024-08-08T20:48:49.4204449Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_size_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4205159Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_size_native.h 2024-08-08T20:48:49.4206155Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_size_ops.h 2024-08-08T20:48:49.4207141Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_softmax_with_shape.h 2024-08-08T20:48:49.4208136Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_softmax_with_shape_native.h 2024-08-08T20:48:49.4209074Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_softmax_with_shape_ops.h 2024-08-08T20:48:49.4209958Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_storage_offsets.h 2024-08-08T20:48:49.4211214Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_storage_offsets_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4211855Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_storage_offsets_native.h 2024-08-08T20:48:49.4212875Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_storage_offsets_ops.h 2024-08-08T20:48:49.4213717Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_strides.h 2024-08-08T20:48:49.4214946Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_strides_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4215658Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_strides_native.h 2024-08-08T20:48:49.4216678Z inflating: build/lib/torch/include/ATen/ops/_nested_tensor_strides_ops.h 2024-08-08T20:48:49.4217636Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer.h 2024-08-08T20:48:49.4218635Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer_copy.h 2024-08-08T20:48:49.4219920Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4220994Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4221690Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer_copy_native.h 2024-08-08T20:48:49.4222721Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer_copy_ops.h 2024-08-08T20:48:49.4223684Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer_cpu_dispatch.h 2024-08-08T20:48:49.4224667Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer_cuda_dispatch.h 2024-08-08T20:48:49.4225525Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer_native.h 2024-08-08T20:48:49.4226481Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_buffer_ops.h 2024-08-08T20:48:49.4227703Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_jagged.h 2024-08-08T20:48:49.4229463Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_jagged_copy.h 2024-08-08T20:48:49.4230359Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_jagged_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4231552Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_jagged_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4232272Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_jagged_copy_native.h 2024-08-08T20:48:49.4233405Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_jagged_copy_ops.h 2024-08-08T20:48:49.4234279Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_jagged_native.h 2024-08-08T20:48:49.4235306Z inflating: build/lib/torch/include/ATen/ops/_nested_view_from_jagged_ops.h 2024-08-08T20:48:49.4236311Z inflating: build/lib/torch/include/ATen/ops/_new_zeros_with_same_feature_meta.h 2024-08-08T20:48:49.4237573Z inflating: build/lib/torch/include/ATen/ops/_new_zeros_with_same_feature_meta_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4238284Z inflating: build/lib/torch/include/ATen/ops/_new_zeros_with_same_feature_meta_native.h 2024-08-08T20:48:49.4239305Z inflating: build/lib/torch/include/ATen/ops/_new_zeros_with_same_feature_meta_ops.h 2024-08-08T20:48:49.4240388Z inflating: build/lib/torch/include/ATen/ops/_nnpack_available_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4241059Z inflating: build/lib/torch/include/ATen/ops/_nnpack_available_native.h 2024-08-08T20:48:49.4242013Z inflating: build/lib/torch/include/ATen/ops/_nnpack_available_ops.h 2024-08-08T20:48:49.4243564Z inflating: build/lib/torch/include/ATen/ops/_nnpack_spatial_convolution.h 2024-08-08T20:48:49.4244640Z inflating: build/lib/torch/include/ATen/ops/_nnpack_spatial_convolution_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4245464Z inflating: build/lib/torch/include/ATen/ops/_nnpack_spatial_convolution_native.h 2024-08-08T20:48:49.4246525Z inflating: build/lib/torch/include/ATen/ops/_nnpack_spatial_convolution_ops.h 2024-08-08T20:48:49.4247271Z inflating: build/lib/torch/include/ATen/ops/_nnz.h 2024-08-08T20:48:49.4248220Z inflating: build/lib/torch/include/ATen/ops/_nnz_native.h 2024-08-08T20:48:49.4249247Z inflating: build/lib/torch/include/ATen/ops/_nnz_ops.h 2024-08-08T20:48:49.4250318Z inflating: build/lib/torch/include/ATen/ops/_pack_padded_sequence.h 2024-08-08T20:48:49.4251416Z inflating: build/lib/torch/include/ATen/ops/_pack_padded_sequence_backward.h 2024-08-08T20:48:49.4252656Z inflating: build/lib/torch/include/ATen/ops/_pack_padded_sequence_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4253444Z inflating: build/lib/torch/include/ATen/ops/_pack_padded_sequence_backward_native.h 2024-08-08T20:48:49.4254385Z inflating: build/lib/torch/include/ATen/ops/_pack_padded_sequence_backward_ops.h 2024-08-08T20:48:49.4255525Z inflating: build/lib/torch/include/ATen/ops/_pack_padded_sequence_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4256247Z inflating: build/lib/torch/include/ATen/ops/_pack_padded_sequence_native.h 2024-08-08T20:48:49.4257286Z inflating: build/lib/torch/include/ATen/ops/_pack_padded_sequence_ops.h 2024-08-08T20:48:49.4258171Z inflating: build/lib/torch/include/ATen/ops/_pad_circular.h 2024-08-08T20:48:49.4259701Z inflating: build/lib/torch/include/ATen/ops/_pad_circular_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4260652Z inflating: build/lib/torch/include/ATen/ops/_pad_circular_native.h 2024-08-08T20:48:49.4261771Z inflating: build/lib/torch/include/ATen/ops/_pad_circular_ops.h 2024-08-08T20:48:49.4262701Z inflating: build/lib/torch/include/ATen/ops/_pad_enum.h 2024-08-08T20:48:49.4264088Z inflating: build/lib/torch/include/ATen/ops/_pad_enum_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4264723Z inflating: build/lib/torch/include/ATen/ops/_pad_enum_native.h 2024-08-08T20:48:49.4265665Z inflating: build/lib/torch/include/ATen/ops/_pad_enum_ops.h 2024-08-08T20:48:49.4266663Z inflating: build/lib/torch/include/ATen/ops/_pad_packed_sequence.h 2024-08-08T20:48:49.4267887Z inflating: build/lib/torch/include/ATen/ops/_pad_packed_sequence_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4268602Z inflating: build/lib/torch/include/ATen/ops/_pad_packed_sequence_native.h 2024-08-08T20:48:49.4269590Z inflating: build/lib/torch/include/ATen/ops/_pad_packed_sequence_ops.h 2024-08-08T20:48:49.4270657Z inflating: build/lib/torch/include/ATen/ops/_padded_dense_to_jagged_forward.h 2024-08-08T20:48:49.4271739Z inflating: build/lib/torch/include/ATen/ops/_padded_dense_to_jagged_forward_cpu_dispatch.h 2024-08-08T20:48:49.4272714Z inflating: build/lib/torch/include/ATen/ops/_padded_dense_to_jagged_forward_cuda_dispatch.h 2024-08-08T20:48:49.4273609Z inflating: build/lib/torch/include/ATen/ops/_padded_dense_to_jagged_forward_native.h 2024-08-08T20:48:49.4274603Z inflating: build/lib/torch/include/ATen/ops/_padded_dense_to_jagged_forward_ops.h 2024-08-08T20:48:49.4275463Z inflating: build/lib/torch/include/ATen/ops/_pdist_backward.h 2024-08-08T20:48:49.4276720Z inflating: build/lib/torch/include/ATen/ops/_pdist_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4277498Z inflating: build/lib/torch/include/ATen/ops/_pdist_backward_cpu_dispatch.h 2024-08-08T20:48:49.4278426Z inflating: build/lib/torch/include/ATen/ops/_pdist_backward_cuda_dispatch.h 2024-08-08T20:48:49.4279291Z inflating: build/lib/torch/include/ATen/ops/_pdist_backward_native.h 2024-08-08T20:48:49.4280304Z inflating: build/lib/torch/include/ATen/ops/_pdist_backward_ops.h 2024-08-08T20:48:49.4281315Z inflating: build/lib/torch/include/ATen/ops/_pdist_forward.h 2024-08-08T20:48:49.4282525Z inflating: build/lib/torch/include/ATen/ops/_pdist_forward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4283308Z inflating: build/lib/torch/include/ATen/ops/_pdist_forward_cpu_dispatch.h 2024-08-08T20:48:49.4284221Z inflating: build/lib/torch/include/ATen/ops/_pdist_forward_cuda_dispatch.h 2024-08-08T20:48:49.4285092Z inflating: build/lib/torch/include/ATen/ops/_pdist_forward_native.h 2024-08-08T20:48:49.4286082Z inflating: build/lib/torch/include/ATen/ops/_pdist_forward_ops.h 2024-08-08T20:48:49.4287029Z inflating: build/lib/torch/include/ATen/ops/_pin_memory.h 2024-08-08T20:48:49.4288299Z inflating: build/lib/torch/include/ATen/ops/_pin_memory_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4289024Z inflating: build/lib/torch/include/ATen/ops/_pin_memory_native.h 2024-08-08T20:48:49.4290107Z inflating: build/lib/torch/include/ATen/ops/_pin_memory_ops.h 2024-08-08T20:48:49.4291005Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel.h 2024-08-08T20:48:49.4292182Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel_backward.h 2024-08-08T20:48:49.4294129Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel_backward_cpu_dispatch.h 2024-08-08T20:48:49.4294720Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel_backward_cuda_dispatch.h 2024-08-08T20:48:49.4295691Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel_backward_native.h 2024-08-08T20:48:49.4296750Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel_backward_ops.h 2024-08-08T20:48:49.4297689Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel_cpu_dispatch.h 2024-08-08T20:48:49.4298630Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel_cuda_dispatch.h 2024-08-08T20:48:49.4299480Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel_native.h 2024-08-08T20:48:49.4300427Z inflating: build/lib/torch/include/ATen/ops/_prelu_kernel_ops.h 2024-08-08T20:48:49.4301334Z inflating: build/lib/torch/include/ATen/ops/_print.h 2024-08-08T20:48:49.4302680Z inflating: build/lib/torch/include/ATen/ops/_print_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4303258Z inflating: build/lib/torch/include/ATen/ops/_print_native.h 2024-08-08T20:48:49.4304291Z inflating: build/lib/torch/include/ATen/ops/_print_ops.h 2024-08-08T20:48:49.4305235Z inflating: build/lib/torch/include/ATen/ops/_propagate_xla_data.h 2024-08-08T20:48:49.4306414Z inflating: build/lib/torch/include/ATen/ops/_propagate_xla_data_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4307117Z inflating: build/lib/torch/include/ATen/ops/_propagate_xla_data_native.h 2024-08-08T20:48:49.4308053Z inflating: build/lib/torch/include/ATen/ops/_propagate_xla_data_ops.h 2024-08-08T20:48:49.4308945Z inflating: build/lib/torch/include/ATen/ops/_remove_batch_dim.h 2024-08-08T20:48:49.4310196Z inflating: build/lib/torch/include/ATen/ops/_remove_batch_dim_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4310872Z inflating: build/lib/torch/include/ATen/ops/_remove_batch_dim_native.h 2024-08-08T20:48:49.4311842Z inflating: build/lib/torch/include/ATen/ops/_remove_batch_dim_ops.h 2024-08-08T20:48:49.4312823Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias.h 2024-08-08T20:48:49.4314201Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_copy.h 2024-08-08T20:48:49.4315271Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4316423Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4316996Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_copy_native.h 2024-08-08T20:48:49.4317967Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_copy_ops.h 2024-08-08T20:48:49.4318999Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_cpu_dispatch.h 2024-08-08T20:48:49.4319941Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_cuda_dispatch.h 2024-08-08T20:48:49.4320938Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_meta_dispatch.h 2024-08-08T20:48:49.4321863Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_native.h 2024-08-08T20:48:49.4322849Z inflating: build/lib/torch/include/ATen/ops/_reshape_alias_ops.h 2024-08-08T20:48:49.4324323Z inflating: build/lib/torch/include/ATen/ops/_reshape_copy.h 2024-08-08T20:48:49.4325214Z inflating: build/lib/torch/include/ATen/ops/_reshape_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4325923Z inflating: build/lib/torch/include/ATen/ops/_reshape_copy_native.h 2024-08-08T20:48:49.4326869Z inflating: build/lib/torch/include/ATen/ops/_reshape_copy_ops.h 2024-08-08T20:48:49.4327783Z inflating: build/lib/torch/include/ATen/ops/_reshape_from_tensor.h 2024-08-08T20:48:49.4329135Z inflating: build/lib/torch/include/ATen/ops/_reshape_from_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4329815Z inflating: build/lib/torch/include/ATen/ops/_reshape_from_tensor_native.h 2024-08-08T20:48:49.4330756Z inflating: build/lib/torch/include/ATen/ops/_reshape_from_tensor_ops.h 2024-08-08T20:48:49.4331879Z inflating: build/lib/torch/include/ATen/ops/_resize_output.h 2024-08-08T20:48:49.4333151Z inflating: build/lib/torch/include/ATen/ops/_resize_output_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4333988Z inflating: build/lib/torch/include/ATen/ops/_resize_output_meta_dispatch.h 2024-08-08T20:48:49.4334876Z inflating: build/lib/torch/include/ATen/ops/_resize_output_native.h 2024-08-08T20:48:49.4335929Z inflating: build/lib/torch/include/ATen/ops/_resize_output_ops.h 2024-08-08T20:48:49.4336859Z inflating: build/lib/torch/include/ATen/ops/_rowwise_prune.h 2024-08-08T20:48:49.4338124Z inflating: build/lib/torch/include/ATen/ops/_rowwise_prune_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4338853Z inflating: build/lib/torch/include/ATen/ops/_rowwise_prune_native.h 2024-08-08T20:48:49.4339878Z inflating: build/lib/torch/include/ATen/ops/_rowwise_prune_ops.h 2024-08-08T20:48:49.4340773Z inflating: build/lib/torch/include/ATen/ops/_safe_softmax.h 2024-08-08T20:48:49.4342004Z inflating: build/lib/torch/include/ATen/ops/_safe_softmax_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4342677Z inflating: build/lib/torch/include/ATen/ops/_safe_softmax_native.h 2024-08-08T20:48:49.4343646Z inflating: build/lib/torch/include/ATen/ops/_safe_softmax_ops.h 2024-08-08T20:48:49.4344689Z inflating: build/lib/torch/include/ATen/ops/_sample_dirichlet.h 2024-08-08T20:48:49.4345947Z inflating: build/lib/torch/include/ATen/ops/_sample_dirichlet_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4346776Z inflating: build/lib/torch/include/ATen/ops/_sample_dirichlet_cpu_dispatch.h 2024-08-08T20:48:49.4347719Z inflating: build/lib/torch/include/ATen/ops/_sample_dirichlet_cuda_dispatch.h 2024-08-08T20:48:49.4348643Z inflating: build/lib/torch/include/ATen/ops/_sample_dirichlet_native.h 2024-08-08T20:48:49.4349618Z inflating: build/lib/torch/include/ATen/ops/_sample_dirichlet_ops.h 2024-08-08T20:48:49.4350568Z inflating: build/lib/torch/include/ATen/ops/_saturate_weight_to_fp16.h 2024-08-08T20:48:49.4351844Z inflating: build/lib/torch/include/ATen/ops/_saturate_weight_to_fp16_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4352525Z inflating: build/lib/torch/include/ATen/ops/_saturate_weight_to_fp16_native.h 2024-08-08T20:48:49.4353478Z inflating: build/lib/torch/include/ATen/ops/_saturate_weight_to_fp16_ops.h 2024-08-08T20:48:49.4354516Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_attention_math.h 2024-08-08T20:48:49.4355973Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_attention_math_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4357295Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_attention_math_for_mps.h 2024-08-08T20:48:49.4358235Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_attention_math_for_mps_native.h 2024-08-08T20:48:49.4359527Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_attention_math_for_mps_ops.h 2024-08-08T20:48:49.4360452Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_attention_math_native.h 2024-08-08T20:48:49.4361481Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_attention_math_ops.h 2024-08-08T20:48:49.4362511Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_cudnn_attention.h 2024-08-08T20:48:49.4363809Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_cudnn_attention_backward.h 2024-08-08T20:48:49.4364936Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_cudnn_attention_backward_cuda_dispatch.h 2024-08-08T20:48:49.4365859Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_cudnn_attention_backward_native.h 2024-08-08T20:48:49.4366933Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_cudnn_attention_backward_ops.h 2024-08-08T20:48:49.4367909Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_cudnn_attention_cuda_dispatch.h 2024-08-08T20:48:49.4368832Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_cudnn_attention_native.h 2024-08-08T20:48:49.4369896Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_cudnn_attention_ops.h 2024-08-08T20:48:49.4370856Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_efficient_attention.h 2024-08-08T20:48:49.4372021Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_efficient_attention_backward.h 2024-08-08T20:48:49.4373097Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_efficient_attention_backward_cuda_dispatch.h 2024-08-08T20:48:49.4374015Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_efficient_attention_backward_native.h 2024-08-08T20:48:49.4375209Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_efficient_attention_backward_ops.h 2024-08-08T20:48:49.4376123Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_efficient_attention_cuda_dispatch.h 2024-08-08T20:48:49.4377026Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_efficient_attention_native.h 2024-08-08T20:48:49.4378091Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_efficient_attention_ops.h 2024-08-08T20:48:49.4379044Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention.h 2024-08-08T20:48:49.4380261Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_backward.h 2024-08-08T20:48:49.4381403Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_backward_cuda_dispatch.h 2024-08-08T20:48:49.4382419Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_backward_native.h 2024-08-08T20:48:49.4383507Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_backward_ops.h 2024-08-08T20:48:49.4384527Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_cuda_dispatch.h 2024-08-08T20:48:49.4385511Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_for_cpu.h 2024-08-08T20:48:49.4386671Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_for_cpu_backward.h 2024-08-08T20:48:49.4387720Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_for_cpu_backward_cpu_dispatch.h 2024-08-08T20:48:49.4388647Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_for_cpu_backward_native.h 2024-08-08T20:48:49.4389933Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_for_cpu_backward_ops.h 2024-08-08T20:48:49.4390868Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_for_cpu_cpu_dispatch.h 2024-08-08T20:48:49.4391808Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_for_cpu_native.h 2024-08-08T20:48:49.4392993Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_for_cpu_ops.h 2024-08-08T20:48:49.4393943Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_native.h 2024-08-08T20:48:49.4394970Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_flash_attention_ops.h 2024-08-08T20:48:49.4396064Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_fused_attention_overrideable.h 2024-08-08T20:48:49.4397400Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_fused_attention_overrideable_backward.h 2024-08-08T20:48:49.4398726Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_fused_attention_overrideable_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4399668Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_fused_attention_overrideable_backward_native.h 2024-08-08T20:48:49.4400608Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_fused_attention_overrideable_backward_ops.h 2024-08-08T20:48:49.4401768Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_fused_attention_overrideable_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4402547Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_fused_attention_overrideable_native.h 2024-08-08T20:48:49.4403578Z inflating: build/lib/torch/include/ATen/ops/_scaled_dot_product_fused_attention_overrideable_ops.h 2024-08-08T20:48:49.4404342Z inflating: build/lib/torch/include/ATen/ops/_scaled_mm.h 2024-08-08T20:48:49.4405440Z inflating: build/lib/torch/include/ATen/ops/_scaled_mm_cuda_dispatch.h 2024-08-08T20:48:49.4406331Z inflating: build/lib/torch/include/ATen/ops/_scaled_mm_native.h 2024-08-08T20:48:49.4407398Z inflating: build/lib/torch/include/ATen/ops/_scaled_mm_ops.h 2024-08-08T20:48:49.4408612Z inflating: build/lib/torch/include/ATen/ops/_segment_reduce_backward.h 2024-08-08T20:48:49.4409840Z inflating: build/lib/torch/include/ATen/ops/_segment_reduce_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4410655Z inflating: build/lib/torch/include/ATen/ops/_segment_reduce_backward_cpu_dispatch.h 2024-08-08T20:48:49.4411736Z inflating: build/lib/torch/include/ATen/ops/_segment_reduce_backward_cuda_dispatch.h 2024-08-08T20:48:49.4412653Z inflating: build/lib/torch/include/ATen/ops/_segment_reduce_backward_native.h 2024-08-08T20:48:49.4413738Z inflating: build/lib/torch/include/ATen/ops/_segment_reduce_backward_ops.h 2024-08-08T20:48:49.4414595Z inflating: build/lib/torch/include/ATen/ops/_shape_as_tensor.h 2024-08-08T20:48:49.4415887Z inflating: build/lib/torch/include/ATen/ops/_shape_as_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4416586Z inflating: build/lib/torch/include/ATen/ops/_shape_as_tensor_native.h 2024-08-08T20:48:49.4417521Z inflating: build/lib/torch/include/ATen/ops/_shape_as_tensor_ops.h 2024-08-08T20:48:49.4419424Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_backward.h 2024-08-08T20:48:49.4420383Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4421382Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.4422434Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.4423594Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_backward_native.h 2024-08-08T20:48:49.4425642Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_backward_ops.h 2024-08-08T20:48:49.4426657Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_forward.h 2024-08-08T20:48:49.4427770Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_forward_cpu_dispatch.h 2024-08-08T20:48:49.4428891Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_forward_cuda_dispatch.h 2024-08-08T20:48:49.4429791Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_forward_native.h 2024-08-08T20:48:49.4430843Z inflating: build/lib/torch/include/ATen/ops/_slow_conv2d_forward_ops.h 2024-08-08T20:48:49.4431742Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_draw.h 2024-08-08T20:48:49.4432991Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_draw_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4433717Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_draw_native.h 2024-08-08T20:48:49.4434710Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_draw_ops.h 2024-08-08T20:48:49.4435612Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_ff.h 2024-08-08T20:48:49.4436929Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_ff_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4437616Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_ff_native.h 2024-08-08T20:48:49.4438619Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_ff_ops.h 2024-08-08T20:48:49.4439638Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_initialize_state.h 2024-08-08T20:48:49.4440878Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_initialize_state_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4441644Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_initialize_state_native.h 2024-08-08T20:48:49.4442618Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_initialize_state_ops.h 2024-08-08T20:48:49.4443514Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_scramble.h 2024-08-08T20:48:49.4444744Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_scramble_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4445534Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_scramble_native.h 2024-08-08T20:48:49.4446489Z inflating: build/lib/torch/include/ATen/ops/_sobol_engine_scramble_ops.h 2024-08-08T20:48:49.4447437Z inflating: build/lib/torch/include/ATen/ops/_softmax.h 2024-08-08T20:48:49.4448485Z inflating: build/lib/torch/include/ATen/ops/_softmax_backward_data.h 2024-08-08T20:48:49.4449818Z inflating: build/lib/torch/include/ATen/ops/_softmax_backward_data_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4450611Z inflating: build/lib/torch/include/ATen/ops/_softmax_backward_data_cpu_dispatch.h 2024-08-08T20:48:49.4451593Z inflating: build/lib/torch/include/ATen/ops/_softmax_backward_data_cuda_dispatch.h 2024-08-08T20:48:49.4452496Z inflating: build/lib/torch/include/ATen/ops/_softmax_backward_data_meta.h 2024-08-08T20:48:49.4453568Z inflating: build/lib/torch/include/ATen/ops/_softmax_backward_data_meta_dispatch.h 2024-08-08T20:48:49.4454471Z inflating: build/lib/torch/include/ATen/ops/_softmax_backward_data_native.h 2024-08-08T20:48:49.4455563Z inflating: build/lib/torch/include/ATen/ops/_softmax_backward_data_ops.h 2024-08-08T20:48:49.4457639Z inflating: build/lib/torch/include/ATen/ops/_softmax_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4458135Z inflating: build/lib/torch/include/ATen/ops/_softmax_cpu_dispatch.h 2024-08-08T20:48:49.4459456Z inflating: build/lib/torch/include/ATen/ops/_softmax_cuda_dispatch.h 2024-08-08T20:48:49.4460365Z inflating: build/lib/torch/include/ATen/ops/_softmax_meta.h 2024-08-08T20:48:49.4461429Z inflating: build/lib/torch/include/ATen/ops/_softmax_meta_dispatch.h 2024-08-08T20:48:49.4462309Z inflating: build/lib/torch/include/ATen/ops/_softmax_native.h 2024-08-08T20:48:49.4463314Z inflating: build/lib/torch/include/ATen/ops/_softmax_ops.h 2024-08-08T20:48:49.4464306Z inflating: build/lib/torch/include/ATen/ops/_sparse_addmm.h 2024-08-08T20:48:49.4465590Z inflating: build/lib/torch/include/ATen/ops/_sparse_addmm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4466436Z inflating: build/lib/torch/include/ATen/ops/_sparse_addmm_native.h 2024-08-08T20:48:49.4467468Z inflating: build/lib/torch/include/ATen/ops/_sparse_addmm_ops.h 2024-08-08T20:48:49.4468429Z inflating: build/lib/torch/include/ATen/ops/_sparse_broadcast_to.h 2024-08-08T20:48:49.4469465Z inflating: build/lib/torch/include/ATen/ops/_sparse_broadcast_to_copy.h 2024-08-08T20:48:49.4470727Z inflating: build/lib/torch/include/ATen/ops/_sparse_broadcast_to_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4471833Z inflating: build/lib/torch/include/ATen/ops/_sparse_broadcast_to_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4472468Z inflating: build/lib/torch/include/ATen/ops/_sparse_broadcast_to_copy_native.h 2024-08-08T20:48:49.4473474Z inflating: build/lib/torch/include/ATen/ops/_sparse_broadcast_to_copy_ops.h 2024-08-08T20:48:49.4474408Z inflating: build/lib/torch/include/ATen/ops/_sparse_broadcast_to_native.h 2024-08-08T20:48:49.4475350Z inflating: build/lib/torch/include/ATen/ops/_sparse_broadcast_to_ops.h 2024-08-08T20:48:49.4476382Z inflating: build/lib/torch/include/ATen/ops/_sparse_bsc_tensor_unsafe.h 2024-08-08T20:48:49.4477657Z inflating: build/lib/torch/include/ATen/ops/_sparse_bsc_tensor_unsafe_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4478510Z inflating: build/lib/torch/include/ATen/ops/_sparse_bsc_tensor_unsafe_native.h 2024-08-08T20:48:49.4479529Z inflating: build/lib/torch/include/ATen/ops/_sparse_bsc_tensor_unsafe_ops.h 2024-08-08T20:48:49.4480593Z inflating: build/lib/torch/include/ATen/ops/_sparse_bsr_tensor_unsafe.h 2024-08-08T20:48:49.4481830Z inflating: build/lib/torch/include/ATen/ops/_sparse_bsr_tensor_unsafe_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4482576Z inflating: build/lib/torch/include/ATen/ops/_sparse_bsr_tensor_unsafe_native.h 2024-08-08T20:48:49.4483625Z inflating: build/lib/torch/include/ATen/ops/_sparse_bsr_tensor_unsafe_ops.h 2024-08-08T20:48:49.4485007Z inflating: build/lib/torch/include/ATen/ops/_sparse_compressed_tensor_unsafe.h 2024-08-08T20:48:49.4486216Z inflating: build/lib/torch/include/ATen/ops/_sparse_compressed_tensor_unsafe_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4486900Z inflating: build/lib/torch/include/ATen/ops/_sparse_compressed_tensor_unsafe_native.h 2024-08-08T20:48:49.4487966Z inflating: build/lib/torch/include/ATen/ops/_sparse_compressed_tensor_unsafe_ops.h 2024-08-08T20:48:49.4488915Z inflating: build/lib/torch/include/ATen/ops/_sparse_compressed_tensor_with_dims.h 2024-08-08T20:48:49.4490457Z inflating: build/lib/torch/include/ATen/ops/_sparse_compressed_tensor_with_dims_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4492154Z inflating: build/lib/torch/include/ATen/ops/_sparse_compressed_tensor_with_dims_native.h 2024-08-08T20:48:49.4492963Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_unsafe.h 2024-08-08T20:48:49.4494596Z inflating: build/lib/torch/include/ATen/ops/_sparse_compressed_tensor_with_dims_ops.h 2024-08-08T20:48:49.4495632Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_unsafe_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4496346Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_unsafe_native.h 2024-08-08T20:48:49.4497429Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_unsafe_ops.h 2024-08-08T20:48:49.4498433Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims.h 2024-08-08T20:48:49.4500297Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims_and_tensors.h 2024-08-08T20:48:49.4501245Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims_and_tensors_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4502229Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims_and_tensors_meta_dispatch.h 2024-08-08T20:48:49.4503147Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims_and_tensors_native.h 2024-08-08T20:48:49.4504219Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims_and_tensors_ops.h 2024-08-08T20:48:49.4505407Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4506288Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims_meta_dispatch.h 2024-08-08T20:48:49.4507174Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims_native.h 2024-08-08T20:48:49.4508220Z inflating: build/lib/torch/include/ATen/ops/_sparse_coo_tensor_with_dims_ops.h 2024-08-08T20:48:49.4509256Z inflating: build/lib/torch/include/ATen/ops/_sparse_csc_tensor_unsafe.h 2024-08-08T20:48:49.4510511Z inflating: build/lib/torch/include/ATen/ops/_sparse_csc_tensor_unsafe_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4511280Z inflating: build/lib/torch/include/ATen/ops/_sparse_csc_tensor_unsafe_native.h 2024-08-08T20:48:49.4512265Z inflating: build/lib/torch/include/ATen/ops/_sparse_csc_tensor_unsafe_ops.h 2024-08-08T20:48:49.4513191Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_prod.h 2024-08-08T20:48:49.4514455Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_prod_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4515194Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_prod_native.h 2024-08-08T20:48:49.4516234Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_prod_ops.h 2024-08-08T20:48:49.4517192Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_sum.h 2024-08-08T20:48:49.4518464Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_sum_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4519182Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_sum_native.h 2024-08-08T20:48:49.4520231Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_sum_ops.h 2024-08-08T20:48:49.4521267Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_tensor_unsafe.h 2024-08-08T20:48:49.4522619Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_tensor_unsafe_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4523256Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_tensor_unsafe_native.h 2024-08-08T20:48:49.4524407Z inflating: build/lib/torch/include/ATen/ops/_sparse_csr_tensor_unsafe_ops.h 2024-08-08T20:48:49.4525353Z inflating: build/lib/torch/include/ATen/ops/_sparse_log_softmax.h 2024-08-08T20:48:49.4526439Z inflating: build/lib/torch/include/ATen/ops/_sparse_log_softmax_backward_data.h 2024-08-08T20:48:49.4527703Z inflating: build/lib/torch/include/ATen/ops/_sparse_log_softmax_backward_data_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4528418Z inflating: build/lib/torch/include/ATen/ops/_sparse_log_softmax_backward_data_native.h 2024-08-08T20:48:49.4529458Z inflating: build/lib/torch/include/ATen/ops/_sparse_log_softmax_backward_data_ops.h 2024-08-08T20:48:49.4530564Z inflating: build/lib/torch/include/ATen/ops/_sparse_log_softmax_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4531572Z inflating: build/lib/torch/include/ATen/ops/_sparse_log_softmax_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4532324Z inflating: build/lib/torch/include/ATen/ops/_sparse_log_softmax_native.h 2024-08-08T20:48:49.4533390Z inflating: build/lib/torch/include/ATen/ops/_sparse_log_softmax_ops.h 2024-08-08T20:48:49.4534333Z inflating: build/lib/torch/include/ATen/ops/_sparse_mask_projection.h 2024-08-08T20:48:49.4535535Z inflating: build/lib/torch/include/ATen/ops/_sparse_mask_projection_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4536246Z inflating: build/lib/torch/include/ATen/ops/_sparse_mask_projection_native.h 2024-08-08T20:48:49.4537274Z inflating: build/lib/torch/include/ATen/ops/_sparse_mask_projection_ops.h 2024-08-08T20:48:49.4538063Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm.h 2024-08-08T20:48:49.4539318Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4539989Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm_native.h 2024-08-08T20:48:49.4540974Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm_ops.h 2024-08-08T20:48:49.4541979Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm_reduce_impl.h 2024-08-08T20:48:49.4543003Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm_reduce_impl_backward.h 2024-08-08T20:48:49.4543992Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm_reduce_impl_backward_native.h 2024-08-08T20:48:49.4545057Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm_reduce_impl_backward_ops.h 2024-08-08T20:48:49.4545938Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm_reduce_impl_native.h 2024-08-08T20:48:49.4546989Z inflating: build/lib/torch/include/ATen/ops/_sparse_mm_reduce_impl_ops.h 2024-08-08T20:48:49.4547935Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_addmm.h 2024-08-08T20:48:49.4549095Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_addmm_cuda_dispatch.h 2024-08-08T20:48:49.4549939Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_addmm_native.h 2024-08-08T20:48:49.4550954Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_addmm_ops.h 2024-08-08T20:48:49.4551835Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_apply.h 2024-08-08T20:48:49.4552897Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_apply_cuda_dispatch.h 2024-08-08T20:48:49.4553801Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_apply_dense.h 2024-08-08T20:48:49.4554841Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_apply_dense_cuda_dispatch.h 2024-08-08T20:48:49.4555944Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_apply_dense_native.h 2024-08-08T20:48:49.4557872Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_apply_dense_ops.h 2024-08-08T20:48:49.4558738Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_apply_native.h 2024-08-08T20:48:49.4559688Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_apply_ops.h 2024-08-08T20:48:49.4560720Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_linear.h 2024-08-08T20:48:49.4561868Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_linear_cuda_dispatch.h 2024-08-08T20:48:49.4562747Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_linear_native.h 2024-08-08T20:48:49.4563747Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_linear_ops.h 2024-08-08T20:48:49.4564656Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_mm.h 2024-08-08T20:48:49.4565715Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_mm_cuda_dispatch.h 2024-08-08T20:48:49.4566629Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_mm_native.h 2024-08-08T20:48:49.4567599Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_mm_ops.h 2024-08-08T20:48:49.4568513Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_tile.h 2024-08-08T20:48:49.4569639Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_tile_cuda_dispatch.h 2024-08-08T20:48:49.4570480Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_tile_native.h 2024-08-08T20:48:49.4571492Z inflating: build/lib/torch/include/ATen/ops/_sparse_semi_structured_tile_ops.h 2024-08-08T20:48:49.4572363Z inflating: build/lib/torch/include/ATen/ops/_sparse_softmax.h 2024-08-08T20:48:49.4573484Z inflating: build/lib/torch/include/ATen/ops/_sparse_softmax_backward_data.h 2024-08-08T20:48:49.4574785Z inflating: build/lib/torch/include/ATen/ops/_sparse_softmax_backward_data_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4575500Z inflating: build/lib/torch/include/ATen/ops/_sparse_softmax_backward_data_native.h 2024-08-08T20:48:49.4576540Z inflating: build/lib/torch/include/ATen/ops/_sparse_softmax_backward_data_ops.h 2024-08-08T20:48:49.4577709Z inflating: build/lib/torch/include/ATen/ops/_sparse_softmax_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4578740Z inflating: build/lib/torch/include/ATen/ops/_sparse_softmax_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4579466Z inflating: build/lib/torch/include/ATen/ops/_sparse_softmax_native.h 2024-08-08T20:48:49.4580543Z inflating: build/lib/torch/include/ATen/ops/_sparse_softmax_ops.h 2024-08-08T20:48:49.4581545Z inflating: build/lib/torch/include/ATen/ops/_sparse_sparse_matmul.h 2024-08-08T20:48:49.4582787Z inflating: build/lib/torch/include/ATen/ops/_sparse_sparse_matmul_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4583598Z inflating: build/lib/torch/include/ATen/ops/_sparse_sparse_matmul_native.h 2024-08-08T20:48:49.4584535Z inflating: build/lib/torch/include/ATen/ops/_sparse_sparse_matmul_ops.h 2024-08-08T20:48:49.4585421Z inflating: build/lib/torch/include/ATen/ops/_sparse_sum.h 2024-08-08T20:48:49.4586478Z inflating: build/lib/torch/include/ATen/ops/_sparse_sum_backward.h 2024-08-08T20:48:49.4587699Z inflating: build/lib/torch/include/ATen/ops/_sparse_sum_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4588689Z inflating: build/lib/torch/include/ATen/ops/_sparse_sum_backward_native.h 2024-08-08T20:48:49.4589835Z inflating: build/lib/torch/include/ATen/ops/_sparse_sum_backward_ops.h 2024-08-08T20:48:49.4590934Z inflating: build/lib/torch/include/ATen/ops/_sparse_sum_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4592201Z inflating: build/lib/torch/include/ATen/ops/_sparse_sum_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4592940Z inflating: build/lib/torch/include/ATen/ops/_sparse_sum_native.h 2024-08-08T20:48:49.4594031Z inflating: build/lib/torch/include/ATen/ops/_sparse_sum_ops.h 2024-08-08T20:48:49.4595091Z inflating: build/lib/torch/include/ATen/ops/_spdiags.h 2024-08-08T20:48:49.4596251Z inflating: build/lib/torch/include/ATen/ops/_spdiags_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4597047Z inflating: build/lib/torch/include/ATen/ops/_spdiags_cpu_dispatch.h 2024-08-08T20:48:49.4597917Z inflating: build/lib/torch/include/ATen/ops/_spdiags_native.h 2024-08-08T20:48:49.4598949Z inflating: build/lib/torch/include/ATen/ops/_spdiags_ops.h 2024-08-08T20:48:49.4599832Z inflating: build/lib/torch/include/ATen/ops/_stack.h 2024-08-08T20:48:49.4601090Z inflating: build/lib/torch/include/ATen/ops/_stack_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4601911Z inflating: build/lib/torch/include/ATen/ops/_stack_cpu_dispatch.h 2024-08-08T20:48:49.4602805Z inflating: build/lib/torch/include/ATen/ops/_stack_native.h 2024-08-08T20:48:49.4603823Z inflating: build/lib/torch/include/ATen/ops/_stack_ops.h 2024-08-08T20:48:49.4604816Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma.h 2024-08-08T20:48:49.4606053Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4606843Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_cpu_dispatch.h 2024-08-08T20:48:49.4607791Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_cuda_dispatch.h 2024-08-08T20:48:49.4608692Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_grad.h 2024-08-08T20:48:49.4609969Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_grad_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4610764Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_grad_cpu_dispatch.h 2024-08-08T20:48:49.4611690Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_grad_cuda_dispatch.h 2024-08-08T20:48:49.4612575Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_grad_native.h 2024-08-08T20:48:49.4613602Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_grad_ops.h 2024-08-08T20:48:49.4614502Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_native.h 2024-08-08T20:48:49.4615510Z inflating: build/lib/torch/include/ATen/ops/_standard_gamma_ops.h 2024-08-08T20:48:49.4616534Z inflating: build/lib/torch/include/ATen/ops/_test_ambiguous_defaults.h 2024-08-08T20:48:49.4617777Z inflating: build/lib/torch/include/ATen/ops/_test_ambiguous_defaults_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4618529Z inflating: build/lib/torch/include/ATen/ops/_test_ambiguous_defaults_native.h 2024-08-08T20:48:49.4619499Z inflating: build/lib/torch/include/ATen/ops/_test_ambiguous_defaults_ops.h 2024-08-08T20:48:49.4620740Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch.h 2024-08-08T20:48:49.4622742Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4623459Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4624211Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_native.h 2024-08-08T20:48:49.4625257Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_ops.h 2024-08-08T20:48:49.4626209Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_view.h 2024-08-08T20:48:49.4627410Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_view_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4628205Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_view_copy.h 2024-08-08T20:48:49.4629414Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_view_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4630674Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_view_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4631442Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_view_copy_native.h 2024-08-08T20:48:49.4632313Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_view_copy_ops.h 2024-08-08T20:48:49.4633206Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_view_native.h 2024-08-08T20:48:49.4634205Z inflating: build/lib/torch/include/ATen/ops/_test_autograd_multiple_dispatch_view_ops.h 2024-08-08T20:48:49.4634936Z inflating: build/lib/torch/include/ATen/ops/_test_check_tensor.h 2024-08-08T20:48:49.4636186Z inflating: build/lib/torch/include/ATen/ops/_test_check_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4636879Z inflating: build/lib/torch/include/ATen/ops/_test_check_tensor_native.h 2024-08-08T20:48:49.4637829Z inflating: build/lib/torch/include/ATen/ops/_test_check_tensor_ops.h 2024-08-08T20:48:49.4638808Z inflating: build/lib/torch/include/ATen/ops/_test_functorch_fallback.h 2024-08-08T20:48:49.4640034Z inflating: build/lib/torch/include/ATen/ops/_test_functorch_fallback_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4640962Z inflating: build/lib/torch/include/ATen/ops/_test_functorch_fallback_cpu_dispatch.h 2024-08-08T20:48:49.4641912Z inflating: build/lib/torch/include/ATen/ops/_test_functorch_fallback_native.h 2024-08-08T20:48:49.4642946Z inflating: build/lib/torch/include/ATen/ops/_test_functorch_fallback_ops.h 2024-08-08T20:48:49.4643921Z inflating: build/lib/torch/include/ATen/ops/_test_optional_filled_intlist.h 2024-08-08T20:48:49.4645148Z inflating: build/lib/torch/include/ATen/ops/_test_optional_filled_intlist_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4646008Z inflating: build/lib/torch/include/ATen/ops/_test_optional_filled_intlist_cpu_dispatch.h 2024-08-08T20:48:49.4646887Z inflating: build/lib/torch/include/ATen/ops/_test_optional_filled_intlist_native.h 2024-08-08T20:48:49.4647905Z inflating: build/lib/torch/include/ATen/ops/_test_optional_filled_intlist_ops.h 2024-08-08T20:48:49.4648869Z inflating: build/lib/torch/include/ATen/ops/_test_optional_floatlist.h 2024-08-08T20:48:49.4650094Z inflating: build/lib/torch/include/ATen/ops/_test_optional_floatlist_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4650895Z inflating: build/lib/torch/include/ATen/ops/_test_optional_floatlist_cpu_dispatch.h 2024-08-08T20:48:49.4651760Z inflating: build/lib/torch/include/ATen/ops/_test_optional_floatlist_native.h 2024-08-08T20:48:49.4652958Z inflating: build/lib/torch/include/ATen/ops/_test_optional_floatlist_ops.h 2024-08-08T20:48:49.4653899Z inflating: build/lib/torch/include/ATen/ops/_test_optional_intlist.h 2024-08-08T20:48:49.4655229Z inflating: build/lib/torch/include/ATen/ops/_test_optional_intlist_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4655992Z inflating: build/lib/torch/include/ATen/ops/_test_optional_intlist_cpu_dispatch.h 2024-08-08T20:48:49.4656848Z inflating: build/lib/torch/include/ATen/ops/_test_optional_intlist_native.h 2024-08-08T20:48:49.4657876Z inflating: build/lib/torch/include/ATen/ops/_test_optional_intlist_ops.h 2024-08-08T20:48:49.4659028Z inflating: build/lib/torch/include/ATen/ops/_test_parallel_materialize.h 2024-08-08T20:48:49.4660267Z inflating: build/lib/torch/include/ATen/ops/_test_parallel_materialize_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4661017Z inflating: build/lib/torch/include/ATen/ops/_test_parallel_materialize_native.h 2024-08-08T20:48:49.4661973Z inflating: build/lib/torch/include/ATen/ops/_test_parallel_materialize_ops.h 2024-08-08T20:48:49.4662901Z inflating: build/lib/torch/include/ATen/ops/_test_serialization_subcmul.h 2024-08-08T20:48:49.4664151Z inflating: build/lib/torch/include/ATen/ops/_test_serialization_subcmul_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4665005Z inflating: build/lib/torch/include/ATen/ops/_test_serialization_subcmul_native.h 2024-08-08T20:48:49.4665895Z inflating: build/lib/torch/include/ATen/ops/_test_serialization_subcmul_ops.h 2024-08-08T20:48:49.4666684Z inflating: build/lib/torch/include/ATen/ops/_test_string_default.h 2024-08-08T20:48:49.4667841Z inflating: build/lib/torch/include/ATen/ops/_test_string_default_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4668624Z inflating: build/lib/torch/include/ATen/ops/_test_string_default_native.h 2024-08-08T20:48:49.4669651Z inflating: build/lib/torch/include/ATen/ops/_test_string_default_ops.h 2024-08-08T20:48:49.4670593Z inflating: build/lib/torch/include/ATen/ops/_test_warn_in_autograd.h 2024-08-08T20:48:49.4671814Z inflating: build/lib/torch/include/ATen/ops/_test_warn_in_autograd_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4672549Z inflating: build/lib/torch/include/ATen/ops/_test_warn_in_autograd_native.h 2024-08-08T20:48:49.4673649Z inflating: build/lib/torch/include/ATen/ops/_test_warn_in_autograd_ops.h 2024-08-08T20:48:49.4674711Z inflating: build/lib/torch/include/ATen/ops/_thnn_differentiable_gru_cell_backward.h 2024-08-08T20:48:49.4675966Z inflating: build/lib/torch/include/ATen/ops/_thnn_differentiable_gru_cell_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4676713Z inflating: build/lib/torch/include/ATen/ops/_thnn_differentiable_gru_cell_backward_native.h 2024-08-08T20:48:49.4677735Z inflating: build/lib/torch/include/ATen/ops/_thnn_differentiable_gru_cell_backward_ops.h 2024-08-08T20:48:49.4678702Z inflating: build/lib/torch/include/ATen/ops/_thnn_differentiable_lstm_cell_backward.h 2024-08-08T20:48:49.4679948Z inflating: build/lib/torch/include/ATen/ops/_thnn_differentiable_lstm_cell_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4680690Z inflating: build/lib/torch/include/ATen/ops/_thnn_differentiable_lstm_cell_backward_native.h 2024-08-08T20:48:49.4681742Z inflating: build/lib/torch/include/ATen/ops/_thnn_differentiable_lstm_cell_backward_ops.h 2024-08-08T20:48:49.4682601Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell.h 2024-08-08T20:48:49.4683683Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell_backward.h 2024-08-08T20:48:49.4685138Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4686459Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell_backward_cuda_dispatch.h 2024-08-08T20:48:49.4687375Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell_backward_native.h 2024-08-08T20:48:49.4688453Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell_backward_ops.h 2024-08-08T20:48:49.4689750Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4690576Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell_cuda_dispatch.h 2024-08-08T20:48:49.4691431Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell_native.h 2024-08-08T20:48:49.4692518Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_gru_cell_ops.h 2024-08-08T20:48:49.4693499Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell.h 2024-08-08T20:48:49.4694557Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_backward.h 2024-08-08T20:48:49.4695795Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4696641Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_backward_impl.h 2024-08-08T20:48:49.4697962Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_backward_impl_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4698776Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_backward_impl_cuda_dispatch.h 2024-08-08T20:48:49.4699772Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_backward_impl_native.h 2024-08-08T20:48:49.4700815Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_backward_impl_ops.h 2024-08-08T20:48:49.4701751Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_backward_native.h 2024-08-08T20:48:49.4702788Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_backward_ops.h 2024-08-08T20:48:49.4704007Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4704823Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_cuda_dispatch.h 2024-08-08T20:48:49.4705708Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_native.h 2024-08-08T20:48:49.4706837Z inflating: build/lib/torch/include/ATen/ops/_thnn_fused_lstm_cell_ops.h 2024-08-08T20:48:49.4707799Z inflating: build/lib/torch/include/ATen/ops/_to_copy.h 2024-08-08T20:48:49.4709072Z inflating: build/lib/torch/include/ATen/ops/_to_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4709834Z inflating: build/lib/torch/include/ATen/ops/_to_copy_native.h 2024-08-08T20:48:49.4710950Z inflating: build/lib/torch/include/ATen/ops/_to_copy_ops.h 2024-08-08T20:48:49.4711834Z inflating: build/lib/torch/include/ATen/ops/_to_cpu.h 2024-08-08T20:48:49.4713075Z inflating: build/lib/torch/include/ATen/ops/_to_cpu_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4713787Z inflating: build/lib/torch/include/ATen/ops/_to_cpu_native.h 2024-08-08T20:48:49.4714787Z inflating: build/lib/torch/include/ATen/ops/_to_cpu_ops.h 2024-08-08T20:48:49.4715711Z inflating: build/lib/torch/include/ATen/ops/_to_dense.h 2024-08-08T20:48:49.4717058Z inflating: build/lib/torch/include/ATen/ops/_to_dense_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4717994Z inflating: build/lib/torch/include/ATen/ops/_to_dense_native.h 2024-08-08T20:48:49.4718987Z inflating: build/lib/torch/include/ATen/ops/_to_dense_ops.h 2024-08-08T20:48:49.4720002Z inflating: build/lib/torch/include/ATen/ops/_to_sparse.h 2024-08-08T20:48:49.4720959Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsc.h 2024-08-08T20:48:49.4722247Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsc_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4723054Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsc_cpu_dispatch.h 2024-08-08T20:48:49.4724013Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsc_cuda_dispatch.h 2024-08-08T20:48:49.4724949Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsc_native.h 2024-08-08T20:48:49.4725971Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsc_ops.h 2024-08-08T20:48:49.4726928Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsr.h 2024-08-08T20:48:49.4728178Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsr_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4728986Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsr_cpu_dispatch.h 2024-08-08T20:48:49.4729983Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsr_cuda_dispatch.h 2024-08-08T20:48:49.4730836Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsr_native.h 2024-08-08T20:48:49.4731852Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_bsr_ops.h 2024-08-08T20:48:49.4733058Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4733869Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_cpu_dispatch.h 2024-08-08T20:48:49.4734785Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csc.h 2024-08-08T20:48:49.4736029Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csc_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4736852Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csc_cpu_dispatch.h 2024-08-08T20:48:49.4737800Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csc_cuda_dispatch.h 2024-08-08T20:48:49.4738901Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csc_native.h 2024-08-08T20:48:49.4739857Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csc_ops.h 2024-08-08T20:48:49.4740755Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csr.h 2024-08-08T20:48:49.4742020Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csr_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4742830Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csr_cpu_dispatch.h 2024-08-08T20:48:49.4743802Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csr_cuda_dispatch.h 2024-08-08T20:48:49.4744692Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csr_native.h 2024-08-08T20:48:49.4745685Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_csr_ops.h 2024-08-08T20:48:49.4746728Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_cuda_dispatch.h 2024-08-08T20:48:49.4747642Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_native.h 2024-08-08T20:48:49.4748756Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_ops.h 2024-08-08T20:48:49.4749986Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_semi_structured.h 2024-08-08T20:48:49.4751867Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_semi_structured_cuda_dispatch.h 2024-08-08T20:48:49.4752538Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_semi_structured_native.h 2024-08-08T20:48:49.4753478Z inflating: build/lib/torch/include/ATen/ops/_to_sparse_semi_structured_ops.h 2024-08-08T20:48:49.4754518Z inflating: build/lib/torch/include/ATen/ops/_transform_bias_rescale_qkv.h 2024-08-08T20:48:49.4755809Z inflating: build/lib/torch/include/ATen/ops/_transform_bias_rescale_qkv_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4756601Z inflating: build/lib/torch/include/ATen/ops/_transform_bias_rescale_qkv_cpu_dispatch.h 2024-08-08T20:48:49.4757611Z inflating: build/lib/torch/include/ATen/ops/_transform_bias_rescale_qkv_cuda_dispatch.h 2024-08-08T20:48:49.4758750Z inflating: build/lib/torch/include/ATen/ops/_transform_bias_rescale_qkv_native.h 2024-08-08T20:48:49.4760449Z inflating: build/lib/torch/include/ATen/ops/_transform_bias_rescale_qkv_ops.h 2024-08-08T20:48:49.4761565Z inflating: build/lib/torch/include/ATen/ops/_transformer_encoder_layer_fwd.h 2024-08-08T20:48:49.4762918Z inflating: build/lib/torch/include/ATen/ops/_transformer_encoder_layer_fwd_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4763734Z inflating: build/lib/torch/include/ATen/ops/_transformer_encoder_layer_fwd_cpu_dispatch.h 2024-08-08T20:48:49.4764755Z inflating: build/lib/torch/include/ATen/ops/_transformer_encoder_layer_fwd_cuda_dispatch.h 2024-08-08T20:48:49.4765702Z inflating: build/lib/torch/include/ATen/ops/_transformer_encoder_layer_fwd_native.h 2024-08-08T20:48:49.4766989Z inflating: build/lib/torch/include/ATen/ops/_transformer_encoder_layer_fwd_ops.h 2024-08-08T20:48:49.4767842Z inflating: build/lib/torch/include/ATen/ops/_trilinear.h 2024-08-08T20:48:49.4769116Z inflating: build/lib/torch/include/ATen/ops/_trilinear_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4770274Z inflating: build/lib/torch/include/ATen/ops/_trilinear_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4770961Z inflating: build/lib/torch/include/ATen/ops/_trilinear_native.h 2024-08-08T20:48:49.4772003Z inflating: build/lib/torch/include/ATen/ops/_trilinear_ops.h 2024-08-08T20:48:49.4773203Z inflating: build/lib/torch/include/ATen/ops/_triton_multi_head_attention.h 2024-08-08T20:48:49.4774479Z inflating: build/lib/torch/include/ATen/ops/_triton_multi_head_attention_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4775326Z inflating: build/lib/torch/include/ATen/ops/_triton_multi_head_attention_cuda_dispatch.h 2024-08-08T20:48:49.4776230Z inflating: build/lib/torch/include/ATen/ops/_triton_multi_head_attention_native.h 2024-08-08T20:48:49.4777459Z inflating: build/lib/torch/include/ATen/ops/_triton_multi_head_attention_ops.h 2024-08-08T20:48:49.4778359Z inflating: build/lib/torch/include/ATen/ops/_triton_scaled_dot_attention.h 2024-08-08T20:48:49.4779559Z inflating: build/lib/torch/include/ATen/ops/_triton_scaled_dot_attention_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4780378Z inflating: build/lib/torch/include/ATen/ops/_triton_scaled_dot_attention_cuda_dispatch.h 2024-08-08T20:48:49.4781252Z inflating: build/lib/torch/include/ATen/ops/_triton_scaled_dot_attention_native.h 2024-08-08T20:48:49.4782312Z inflating: build/lib/torch/include/ATen/ops/_triton_scaled_dot_attention_ops.h 2024-08-08T20:48:49.4783167Z inflating: build/lib/torch/include/ATen/ops/_unique.h 2024-08-08T20:48:49.4784636Z inflating: build/lib/torch/include/ATen/ops/_unique2.h 2024-08-08T20:48:49.4785711Z inflating: build/lib/torch/include/ATen/ops/_unique2_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4786518Z inflating: build/lib/torch/include/ATen/ops/_unique2_cpu_dispatch.h 2024-08-08T20:48:49.4787462Z inflating: build/lib/torch/include/ATen/ops/_unique2_cuda_dispatch.h 2024-08-08T20:48:49.4788373Z inflating: build/lib/torch/include/ATen/ops/_unique2_native.h 2024-08-08T20:48:49.4789408Z inflating: build/lib/torch/include/ATen/ops/_unique2_ops.h 2024-08-08T20:48:49.4790629Z inflating: build/lib/torch/include/ATen/ops/_unique_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4791396Z inflating: build/lib/torch/include/ATen/ops/_unique_cpu_dispatch.h 2024-08-08T20:48:49.4792355Z inflating: build/lib/torch/include/ATen/ops/_unique_cuda_dispatch.h 2024-08-08T20:48:49.4793299Z inflating: build/lib/torch/include/ATen/ops/_unique_native.h 2024-08-08T20:48:49.4794327Z inflating: build/lib/torch/include/ATen/ops/_unique_ops.h 2024-08-08T20:48:49.4795484Z inflating: build/lib/torch/include/ATen/ops/_unpack_dual.h 2024-08-08T20:48:49.4796723Z inflating: build/lib/torch/include/ATen/ops/_unpack_dual_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4797481Z inflating: build/lib/torch/include/ATen/ops/_unpack_dual_native.h 2024-08-08T20:48:49.4798482Z inflating: build/lib/torch/include/ATen/ops/_unpack_dual_ops.h 2024-08-08T20:48:49.4799397Z inflating: build/lib/torch/include/ATen/ops/_unsafe_index.h 2024-08-08T20:48:49.4800657Z inflating: build/lib/torch/include/ATen/ops/_unsafe_index_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4801345Z inflating: build/lib/torch/include/ATen/ops/_unsafe_index_native.h 2024-08-08T20:48:49.4802376Z inflating: build/lib/torch/include/ATen/ops/_unsafe_index_ops.h 2024-08-08T20:48:49.4803380Z inflating: build/lib/torch/include/ATen/ops/_unsafe_index_put.h 2024-08-08T20:48:49.4804613Z inflating: build/lib/torch/include/ATen/ops/_unsafe_index_put_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4805416Z inflating: build/lib/torch/include/ATen/ops/_unsafe_index_put_native.h 2024-08-08T20:48:49.4806376Z inflating: build/lib/torch/include/ATen/ops/_unsafe_index_put_ops.h 2024-08-08T20:48:49.4807323Z inflating: build/lib/torch/include/ATen/ops/_unsafe_masked_index.h 2024-08-08T20:48:49.4808532Z inflating: build/lib/torch/include/ATen/ops/_unsafe_masked_index_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4809242Z inflating: build/lib/torch/include/ATen/ops/_unsafe_masked_index_native.h 2024-08-08T20:48:49.4810252Z inflating: build/lib/torch/include/ATen/ops/_unsafe_masked_index_ops.h 2024-08-08T20:48:49.4811271Z inflating: build/lib/torch/include/ATen/ops/_unsafe_masked_index_put_accumulate.h 2024-08-08T20:48:49.4812607Z inflating: build/lib/torch/include/ATen/ops/_unsafe_masked_index_put_accumulate_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4813316Z inflating: build/lib/torch/include/ATen/ops/_unsafe_masked_index_put_accumulate_native.h 2024-08-08T20:48:49.4814326Z inflating: build/lib/torch/include/ATen/ops/_unsafe_masked_index_put_accumulate_ops.h 2024-08-08T20:48:49.4815567Z inflating: build/lib/torch/include/ATen/ops/_unsafe_view.h 2024-08-08T20:48:49.4816730Z inflating: build/lib/torch/include/ATen/ops/_unsafe_view_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4818278Z inflating: build/lib/torch/include/ATen/ops/_unsafe_view_ops.h 2024-08-08T20:48:49.4819135Z inflating: build/lib/torch/include/ATen/ops/_unsafe_view_native.h 2024-08-08T20:48:49.4820685Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa.h 2024-08-08T20:48:49.4821881Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_backward.h 2024-08-08T20:48:49.4823284Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4824070Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_backward_cpu_dispatch.h 2024-08-08T20:48:49.4825110Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_backward_cuda_dispatch.h 2024-08-08T20:48:49.4826041Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_backward_meta.h 2024-08-08T20:48:49.4827127Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_backward_meta_dispatch.h 2024-08-08T20:48:49.4828051Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_backward_native.h 2024-08-08T20:48:49.4829097Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_backward_ops.h 2024-08-08T20:48:49.4830349Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4831225Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4832121Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_cpu_dispatch.h 2024-08-08T20:48:49.4833151Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_cuda_dispatch.h 2024-08-08T20:48:49.4834017Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_meta.h 2024-08-08T20:48:49.4835200Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_meta_dispatch.h 2024-08-08T20:48:49.4836115Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_native.h 2024-08-08T20:48:49.4837188Z inflating: build/lib/torch/include/ATen/ops/_upsample_bicubic2d_aa_ops.h 2024-08-08T20:48:49.4838772Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa.h 2024-08-08T20:48:49.4839944Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_backward.h 2024-08-08T20:48:49.4841304Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4842107Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_backward_cpu_dispatch.h 2024-08-08T20:48:49.4843158Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_backward_cuda_dispatch.h 2024-08-08T20:48:49.4844049Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_backward_meta.h 2024-08-08T20:48:49.4845121Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_backward_meta_dispatch.h 2024-08-08T20:48:49.4846031Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_backward_native.h 2024-08-08T20:48:49.4847115Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_backward_ops.h 2024-08-08T20:48:49.4848466Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4849357Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4850232Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_cpu_dispatch.h 2024-08-08T20:48:49.4851499Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_cuda_dispatch.h 2024-08-08T20:48:49.4852428Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_meta.h 2024-08-08T20:48:49.4853428Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_meta_dispatch.h 2024-08-08T20:48:49.4854346Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_native.h 2024-08-08T20:48:49.4855418Z inflating: build/lib/torch/include/ATen/ops/_upsample_bilinear2d_aa_ops.h 2024-08-08T20:48:49.4856703Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d.h 2024-08-08T20:48:49.4857957Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_backward.h 2024-08-08T20:48:49.4859525Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4860375Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_backward_cpu_dispatch.h 2024-08-08T20:48:49.4861439Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_backward_cuda_dispatch.h 2024-08-08T20:48:49.4862391Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_backward_meta.h 2024-08-08T20:48:49.4863431Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_backward_meta_dispatch.h 2024-08-08T20:48:49.4864372Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_backward_native.h 2024-08-08T20:48:49.4865372Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_backward_ops.h 2024-08-08T20:48:49.4866637Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4867523Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4868353Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_cpu_dispatch.h 2024-08-08T20:48:49.4869400Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_cuda_dispatch.h 2024-08-08T20:48:49.4870264Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_meta.h 2024-08-08T20:48:49.4871314Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_meta_dispatch.h 2024-08-08T20:48:49.4872252Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_native.h 2024-08-08T20:48:49.4873379Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact1d_ops.h 2024-08-08T20:48:49.4874946Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d.h 2024-08-08T20:48:49.4876010Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_backward.h 2024-08-08T20:48:49.4877394Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4878309Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.4879267Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.4880147Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_backward_meta.h 2024-08-08T20:48:49.4881267Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_backward_meta_dispatch.h 2024-08-08T20:48:49.4882206Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_backward_native.h 2024-08-08T20:48:49.4883287Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_backward_ops.h 2024-08-08T20:48:49.4884541Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4885604Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4887168Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_cpu_dispatch.h 2024-08-08T20:48:49.4888200Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_cuda_dispatch.h 2024-08-08T20:48:49.4888962Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_meta.h 2024-08-08T20:48:49.4890054Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_meta_dispatch.h 2024-08-08T20:48:49.4890997Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_native.h 2024-08-08T20:48:49.4892082Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact2d_ops.h 2024-08-08T20:48:49.4893711Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d.h 2024-08-08T20:48:49.4894841Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_backward.h 2024-08-08T20:48:49.4896210Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4897020Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_backward_cpu_dispatch.h 2024-08-08T20:48:49.4898085Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_backward_cuda_dispatch.h 2024-08-08T20:48:49.4899045Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_backward_meta.h 2024-08-08T20:48:49.4900160Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_backward_meta_dispatch.h 2024-08-08T20:48:49.4901113Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_backward_native.h 2024-08-08T20:48:49.4902157Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_backward_ops.h 2024-08-08T20:48:49.4903417Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4904282Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4905170Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_cpu_dispatch.h 2024-08-08T20:48:49.4906220Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_cuda_dispatch.h 2024-08-08T20:48:49.4907066Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_meta.h 2024-08-08T20:48:49.4908220Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_meta_dispatch.h 2024-08-08T20:48:49.4909135Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_native.h 2024-08-08T20:48:49.4910230Z inflating: build/lib/torch/include/ATen/ops/_upsample_nearest_exact3d_ops.h 2024-08-08T20:48:49.4911129Z inflating: build/lib/torch/include/ATen/ops/_use_cudnn_ctc_loss.h 2024-08-08T20:48:49.4912272Z inflating: build/lib/torch/include/ATen/ops/_use_cudnn_ctc_loss_cuda_dispatch.h 2024-08-08T20:48:49.4913132Z inflating: build/lib/torch/include/ATen/ops/_use_cudnn_ctc_loss_native.h 2024-08-08T20:48:49.4914178Z inflating: build/lib/torch/include/ATen/ops/_use_cudnn_ctc_loss_ops.h 2024-08-08T20:48:49.4915136Z inflating: build/lib/torch/include/ATen/ops/_use_cudnn_rnn_flatten_weight.h 2024-08-08T20:48:49.4916401Z inflating: build/lib/torch/include/ATen/ops/_use_cudnn_rnn_flatten_weight_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4917080Z inflating: build/lib/torch/include/ATen/ops/_use_cudnn_rnn_flatten_weight_native.h 2024-08-08T20:48:49.4918070Z inflating: build/lib/torch/include/ATen/ops/_use_cudnn_rnn_flatten_weight_ops.h 2024-08-08T20:48:49.4919184Z inflating: build/lib/torch/include/ATen/ops/_validate_compressed_sparse_indices.h 2024-08-08T20:48:49.4920318Z inflating: build/lib/torch/include/ATen/ops/_validate_compressed_sparse_indices_cpu_dispatch.h 2024-08-08T20:48:49.4921287Z inflating: build/lib/torch/include/ATen/ops/_validate_compressed_sparse_indices_cuda_dispatch.h 2024-08-08T20:48:49.4922173Z inflating: build/lib/torch/include/ATen/ops/_validate_compressed_sparse_indices_native.h 2024-08-08T20:48:49.4923260Z inflating: build/lib/torch/include/ATen/ops/_validate_compressed_sparse_indices_ops.h 2024-08-08T20:48:49.4924111Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_bsc_tensor_args.h 2024-08-08T20:48:49.4925327Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_bsc_tensor_args_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4926024Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_bsc_tensor_args_native.h 2024-08-08T20:48:49.4927040Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_bsc_tensor_args_ops.h 2024-08-08T20:48:49.4927962Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_bsr_tensor_args.h 2024-08-08T20:48:49.4929187Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_bsr_tensor_args_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4929924Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_bsr_tensor_args_native.h 2024-08-08T20:48:49.4930888Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_bsr_tensor_args_ops.h 2024-08-08T20:48:49.4931876Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_compressed_tensor_args.h 2024-08-08T20:48:49.4933150Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_compressed_tensor_args_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4933842Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_compressed_tensor_args_native.h 2024-08-08T20:48:49.4934864Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_compressed_tensor_args_ops.h 2024-08-08T20:48:49.4935773Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_coo_tensor_args.h 2024-08-08T20:48:49.4937030Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_coo_tensor_args_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4937729Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_coo_tensor_args_native.h 2024-08-08T20:48:49.4938717Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_coo_tensor_args_ops.h 2024-08-08T20:48:49.4939719Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_csc_tensor_args.h 2024-08-08T20:48:49.4940968Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_csc_tensor_args_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4941689Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_csc_tensor_args_native.h 2024-08-08T20:48:49.4942652Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_csc_tensor_args_ops.h 2024-08-08T20:48:49.4943565Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_csr_tensor_args.h 2024-08-08T20:48:49.4944855Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_csr_tensor_args_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4945562Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_csr_tensor_args_native.h 2024-08-08T20:48:49.4946592Z inflating: build/lib/torch/include/ATen/ops/_validate_sparse_csr_tensor_args_ops.h 2024-08-08T20:48:49.4947262Z inflating: build/lib/torch/include/ATen/ops/_values.h 2024-08-08T20:48:49.4948254Z inflating: build/lib/torch/include/ATen/ops/_values_copy.h 2024-08-08T20:48:49.4949488Z inflating: build/lib/torch/include/ATen/ops/_values_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4950753Z inflating: build/lib/torch/include/ATen/ops/_values_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.4951932Z inflating: build/lib/torch/include/ATen/ops/_values_copy_native.h 2024-08-08T20:48:49.4952902Z inflating: build/lib/torch/include/ATen/ops/_values_copy_ops.h 2024-08-08T20:48:49.4953805Z inflating: build/lib/torch/include/ATen/ops/_values_native.h 2024-08-08T20:48:49.4954793Z inflating: build/lib/torch/include/ATen/ops/_values_ops.h 2024-08-08T20:48:49.4955699Z inflating: build/lib/torch/include/ATen/ops/_version.h 2024-08-08T20:48:49.4957009Z inflating: build/lib/torch/include/ATen/ops/_version_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4957735Z inflating: build/lib/torch/include/ATen/ops/_version_native.h 2024-08-08T20:48:49.4958800Z inflating: build/lib/torch/include/ATen/ops/_version_ops.h 2024-08-08T20:48:49.4961569Z inflating: build/lib/torch/include/ATen/ops/_weight_int4pack_mm.h 2024-08-08T20:48:49.4962492Z inflating: build/lib/torch/include/ATen/ops/_weight_int4pack_mm_cpu_dispatch.h 2024-08-08T20:48:49.4963392Z inflating: build/lib/torch/include/ATen/ops/_weight_int4pack_mm_cuda_dispatch.h 2024-08-08T20:48:49.4964314Z inflating: build/lib/torch/include/ATen/ops/_weight_int4pack_mm_native.h 2024-08-08T20:48:49.4965296Z inflating: build/lib/torch/include/ATen/ops/_weight_int4pack_mm_ops.h 2024-08-08T20:48:49.4966185Z inflating: build/lib/torch/include/ATen/ops/_weight_int8pack_mm.h 2024-08-08T20:48:49.4967245Z inflating: build/lib/torch/include/ATen/ops/_weight_int8pack_mm_cpu_dispatch.h 2024-08-08T20:48:49.4968122Z inflating: build/lib/torch/include/ATen/ops/_weight_int8pack_mm_native.h 2024-08-08T20:48:49.4969085Z inflating: build/lib/torch/include/ATen/ops/_weight_int8pack_mm_ops.h 2024-08-08T20:48:49.4969935Z inflating: build/lib/torch/include/ATen/ops/_weight_norm.h 2024-08-08T20:48:49.4971145Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4972064Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_differentiable_backward.h 2024-08-08T20:48:49.4973389Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_differentiable_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4974121Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_differentiable_backward_native.h 2024-08-08T20:48:49.4975121Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_differentiable_backward_ops.h 2024-08-08T20:48:49.4975969Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface.h 2024-08-08T20:48:49.4977089Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_backward.h 2024-08-08T20:48:49.4978337Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4979200Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_backward_cpu_dispatch.h 2024-08-08T20:48:49.4980212Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_backward_cuda_dispatch.h 2024-08-08T20:48:49.4981105Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_backward_native.h 2024-08-08T20:48:49.4982182Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_backward_ops.h 2024-08-08T20:48:49.4983315Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4984315Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_cpu_dispatch.h 2024-08-08T20:48:49.4985326Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_cuda_dispatch.h 2024-08-08T20:48:49.4986167Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_native.h 2024-08-08T20:48:49.4987249Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_interface_ops.h 2024-08-08T20:48:49.4988067Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_native.h 2024-08-08T20:48:49.4989025Z inflating: build/lib/torch/include/ATen/ops/_weight_norm_ops.h 2024-08-08T20:48:49.4989901Z inflating: build/lib/torch/include/ATen/ops/abs.h 2024-08-08T20:48:49.4991148Z inflating: build/lib/torch/include/ATen/ops/abs_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.4991978Z inflating: build/lib/torch/include/ATen/ops/abs_cpu_dispatch.h 2024-08-08T20:48:49.4992913Z inflating: build/lib/torch/include/ATen/ops/abs_cuda_dispatch.h 2024-08-08T20:48:49.4994252Z inflating: build/lib/torch/include/ATen/ops/abs_native.h 2024-08-08T20:48:49.4995091Z inflating: build/lib/torch/include/ATen/ops/abs_ops.h 2024-08-08T20:48:49.4996036Z inflating: build/lib/torch/include/ATen/ops/absolute.h 2024-08-08T20:48:49.4997491Z inflating: build/lib/torch/include/ATen/ops/absolute_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.4998051Z inflating: build/lib/torch/include/ATen/ops/absolute_native.h 2024-08-08T20:48:49.4999028Z inflating: build/lib/torch/include/ATen/ops/absolute_ops.h 2024-08-08T20:48:49.4999907Z inflating: build/lib/torch/include/ATen/ops/acos.h 2024-08-08T20:48:49.5001439Z inflating: build/lib/torch/include/ATen/ops/acos_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5002069Z inflating: build/lib/torch/include/ATen/ops/acos_cpu_dispatch.h 2024-08-08T20:48:49.5002987Z inflating: build/lib/torch/include/ATen/ops/acos_cuda_dispatch.h 2024-08-08T20:48:49.5003832Z inflating: build/lib/torch/include/ATen/ops/acos_meta.h 2024-08-08T20:48:49.5004836Z inflating: build/lib/torch/include/ATen/ops/acos_meta_dispatch.h 2024-08-08T20:48:49.5005711Z inflating: build/lib/torch/include/ATen/ops/acos_native.h 2024-08-08T20:48:49.5006771Z inflating: build/lib/torch/include/ATen/ops/acos_ops.h 2024-08-08T20:48:49.5007695Z inflating: build/lib/torch/include/ATen/ops/acosh.h 2024-08-08T20:48:49.5009107Z inflating: build/lib/torch/include/ATen/ops/acosh_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5009733Z inflating: build/lib/torch/include/ATen/ops/acosh_cpu_dispatch.h 2024-08-08T20:48:49.5010673Z inflating: build/lib/torch/include/ATen/ops/acosh_cuda_dispatch.h 2024-08-08T20:48:49.5011530Z inflating: build/lib/torch/include/ATen/ops/acosh_meta.h 2024-08-08T20:48:49.5012515Z inflating: build/lib/torch/include/ATen/ops/acosh_meta_dispatch.h 2024-08-08T20:48:49.5013385Z inflating: build/lib/torch/include/ATen/ops/acosh_native.h 2024-08-08T20:48:49.5014377Z inflating: build/lib/torch/include/ATen/ops/acosh_ops.h 2024-08-08T20:48:49.5015770Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool1d.h 2024-08-08T20:48:49.5017819Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5018608Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool1d_native.h 2024-08-08T20:48:49.5019295Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool1d_ops.h 2024-08-08T20:48:49.5020440Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool2d.h 2024-08-08T20:48:49.5021751Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5022594Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool2d_cpu_dispatch.h 2024-08-08T20:48:49.5023506Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool2d_cuda_dispatch.h 2024-08-08T20:48:49.5024447Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool2d_native.h 2024-08-08T20:48:49.5025493Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool2d_ops.h 2024-08-08T20:48:49.5026529Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d.h 2024-08-08T20:48:49.5027576Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_backward.h 2024-08-08T20:48:49.5028601Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_backward_cpu_dispatch.h 2024-08-08T20:48:49.5029562Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_backward_cuda_dispatch.h 2024-08-08T20:48:49.5030463Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_backward_native.h 2024-08-08T20:48:49.5031498Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_backward_ops.h 2024-08-08T20:48:49.5032662Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5033440Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_cpu_dispatch.h 2024-08-08T20:48:49.5034431Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_cuda_dispatch.h 2024-08-08T20:48:49.5035505Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_native.h 2024-08-08T20:48:49.5036412Z inflating: build/lib/torch/include/ATen/ops/adaptive_avg_pool3d_ops.h 2024-08-08T20:48:49.5037282Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool1d.h 2024-08-08T20:48:49.5038508Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5039215Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool1d_native.h 2024-08-08T20:48:49.5040177Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool1d_ops.h 2024-08-08T20:48:49.5041204Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d.h 2024-08-08T20:48:49.5042267Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_backward.h 2024-08-08T20:48:49.5043600Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5044364Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.5045349Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.5046204Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_backward_meta.h 2024-08-08T20:48:49.5047255Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_backward_meta_dispatch.h 2024-08-08T20:48:49.5048305Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_backward_native.h 2024-08-08T20:48:49.5049353Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_backward_ops.h 2024-08-08T20:48:49.5050619Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5051323Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_cpu_dispatch.h 2024-08-08T20:48:49.5052303Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_cuda_dispatch.h 2024-08-08T20:48:49.5053141Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_meta.h 2024-08-08T20:48:49.5054178Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_meta_dispatch.h 2024-08-08T20:48:49.5055059Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_native.h 2024-08-08T20:48:49.5056082Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool2d_ops.h 2024-08-08T20:48:49.5057026Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d.h 2024-08-08T20:48:49.5058074Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_backward.h 2024-08-08T20:48:49.5059810Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5060526Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_backward_cpu_dispatch.h 2024-08-08T20:48:49.5061531Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_backward_cuda_dispatch.h 2024-08-08T20:48:49.5062328Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_backward_meta.h 2024-08-08T20:48:49.5063410Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_backward_meta_dispatch.h 2024-08-08T20:48:49.5064324Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_backward_native.h 2024-08-08T20:48:49.5065357Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_backward_ops.h 2024-08-08T20:48:49.5066635Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5067313Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_cpu_dispatch.h 2024-08-08T20:48:49.5068299Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_cuda_dispatch.h 2024-08-08T20:48:49.5069185Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_meta.h 2024-08-08T20:48:49.5070254Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_meta_dispatch.h 2024-08-08T20:48:49.5071276Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_native.h 2024-08-08T20:48:49.5072184Z inflating: build/lib/torch/include/ATen/ops/adaptive_max_pool3d_ops.h 2024-08-08T20:48:49.5073048Z inflating: build/lib/torch/include/ATen/ops/add.h 2024-08-08T20:48:49.5074325Z inflating: build/lib/torch/include/ATen/ops/add_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5075410Z inflating: build/lib/torch/include/ATen/ops/add_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5076107Z inflating: build/lib/torch/include/ATen/ops/add_cpu_dispatch.h 2024-08-08T20:48:49.5077066Z inflating: build/lib/torch/include/ATen/ops/add_cuda_dispatch.h 2024-08-08T20:48:49.5077908Z inflating: build/lib/torch/include/ATen/ops/add_meta.h 2024-08-08T20:48:49.5078939Z inflating: build/lib/torch/include/ATen/ops/add_meta_dispatch.h 2024-08-08T20:48:49.5080428Z inflating: build/lib/torch/include/ATen/ops/add_native.h 2024-08-08T20:48:49.5082183Z inflating: build/lib/torch/include/ATen/ops/add_ops.h 2024-08-08T20:48:49.5082787Z inflating: build/lib/torch/include/ATen/ops/addbmm.h 2024-08-08T20:48:49.5084184Z inflating: build/lib/torch/include/ATen/ops/addbmm_cpu_dispatch.h 2024-08-08T20:48:49.5084958Z inflating: build/lib/torch/include/ATen/ops/addbmm_cuda_dispatch.h 2024-08-08T20:48:49.5086001Z inflating: build/lib/torch/include/ATen/ops/addbmm_meta_dispatch.h 2024-08-08T20:48:49.5086942Z inflating: build/lib/torch/include/ATen/ops/addbmm_native.h 2024-08-08T20:48:49.5088010Z inflating: build/lib/torch/include/ATen/ops/addbmm_ops.h 2024-08-08T20:48:49.5088963Z inflating: build/lib/torch/include/ATen/ops/addcdiv.h 2024-08-08T20:48:49.5090372Z inflating: build/lib/torch/include/ATen/ops/addcdiv_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5091068Z inflating: build/lib/torch/include/ATen/ops/addcdiv_cpu_dispatch.h 2024-08-08T20:48:49.5092056Z inflating: build/lib/torch/include/ATen/ops/addcdiv_cuda_dispatch.h 2024-08-08T20:48:49.5092889Z inflating: build/lib/torch/include/ATen/ops/addcdiv_meta.h 2024-08-08T20:48:49.5093948Z inflating: build/lib/torch/include/ATen/ops/addcdiv_meta_dispatch.h 2024-08-08T20:48:49.5094813Z inflating: build/lib/torch/include/ATen/ops/addcdiv_native.h 2024-08-08T20:48:49.5095868Z inflating: build/lib/torch/include/ATen/ops/addcdiv_ops.h 2024-08-08T20:48:49.5096854Z inflating: build/lib/torch/include/ATen/ops/addcmul.h 2024-08-08T20:48:49.5098220Z inflating: build/lib/torch/include/ATen/ops/addcmul_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5098914Z inflating: build/lib/torch/include/ATen/ops/addcmul_cpu_dispatch.h 2024-08-08T20:48:49.5099860Z inflating: build/lib/torch/include/ATen/ops/addcmul_cuda_dispatch.h 2024-08-08T20:48:49.5100754Z inflating: build/lib/torch/include/ATen/ops/addcmul_meta.h 2024-08-08T20:48:49.5101802Z inflating: build/lib/torch/include/ATen/ops/addcmul_meta_dispatch.h 2024-08-08T20:48:49.5102667Z inflating: build/lib/torch/include/ATen/ops/addcmul_native.h 2024-08-08T20:48:49.5103715Z inflating: build/lib/torch/include/ATen/ops/addcmul_ops.h 2024-08-08T20:48:49.5104631Z inflating: build/lib/torch/include/ATen/ops/addmm.h 2024-08-08T20:48:49.5106090Z inflating: build/lib/torch/include/ATen/ops/addmm_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5106793Z inflating: build/lib/torch/include/ATen/ops/addmm_cpu_dispatch.h 2024-08-08T20:48:49.5107740Z inflating: build/lib/torch/include/ATen/ops/addmm_cuda_dispatch.h 2024-08-08T20:48:49.5108656Z inflating: build/lib/torch/include/ATen/ops/addmm_meta.h 2024-08-08T20:48:49.5109729Z inflating: build/lib/torch/include/ATen/ops/addmm_meta_dispatch.h 2024-08-08T20:48:49.5110740Z inflating: build/lib/torch/include/ATen/ops/addmm_native.h 2024-08-08T20:48:49.5111796Z inflating: build/lib/torch/include/ATen/ops/addmm_ops.h 2024-08-08T20:48:49.5113314Z inflating: build/lib/torch/include/ATen/ops/addmv.h 2024-08-08T20:48:49.5114333Z inflating: build/lib/torch/include/ATen/ops/addmv_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5115073Z inflating: build/lib/torch/include/ATen/ops/addmv_cpu_dispatch.h 2024-08-08T20:48:49.5116038Z inflating: build/lib/torch/include/ATen/ops/addmv_cuda_dispatch.h 2024-08-08T20:48:49.5116899Z inflating: build/lib/torch/include/ATen/ops/addmv_meta.h 2024-08-08T20:48:49.5117967Z inflating: build/lib/torch/include/ATen/ops/addmv_meta_dispatch.h 2024-08-08T20:48:49.5118856Z inflating: build/lib/torch/include/ATen/ops/addmv_native.h 2024-08-08T20:48:49.5119949Z inflating: build/lib/torch/include/ATen/ops/addmv_ops.h 2024-08-08T20:48:49.5120873Z inflating: build/lib/torch/include/ATen/ops/addr.h 2024-08-08T20:48:49.5132990Z inflating: build/lib/torch/include/ATen/ops/addr_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5133505Z inflating: build/lib/torch/include/ATen/ops/addr_cpu_dispatch.h 2024-08-08T20:48:49.5134130Z inflating: build/lib/torch/include/ATen/ops/addr_cuda_dispatch.h 2024-08-08T20:48:49.5134483Z inflating: build/lib/torch/include/ATen/ops/addr_native.h 2024-08-08T20:48:49.5134767Z inflating: build/lib/torch/include/ATen/ops/addr_ops.h 2024-08-08T20:48:49.5135043Z inflating: build/lib/torch/include/ATen/ops/adjoint.h 2024-08-08T20:48:49.5135624Z inflating: build/lib/torch/include/ATen/ops/adjoint_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5135962Z inflating: build/lib/torch/include/ATen/ops/adjoint_native.h 2024-08-08T20:48:49.5136259Z inflating: build/lib/torch/include/ATen/ops/adjoint_ops.h 2024-08-08T20:48:49.5136667Z inflating: build/lib/torch/include/ATen/ops/affine_grid_generator.h 2024-08-08T20:48:49.5137132Z inflating: build/lib/torch/include/ATen/ops/affine_grid_generator_backward.h 2024-08-08T20:48:49.5137885Z inflating: build/lib/torch/include/ATen/ops/affine_grid_generator_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5138398Z inflating: build/lib/torch/include/ATen/ops/affine_grid_generator_backward_native.h 2024-08-08T20:48:49.5138886Z inflating: build/lib/torch/include/ATen/ops/affine_grid_generator_backward_ops.h 2024-08-08T20:48:49.5139565Z inflating: build/lib/torch/include/ATen/ops/affine_grid_generator_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5140016Z inflating: build/lib/torch/include/ATen/ops/affine_grid_generator_native.h 2024-08-08T20:48:49.5140441Z inflating: build/lib/torch/include/ATen/ops/affine_grid_generator_ops.h 2024-08-08T20:48:49.5140736Z inflating: build/lib/torch/include/ATen/ops/alias.h 2024-08-08T20:48:49.5141270Z inflating: build/lib/torch/include/ATen/ops/alias_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5141755Z inflating: build/lib/torch/include/ATen/ops/alias_copy.h 2024-08-08T20:48:49.5142339Z inflating: build/lib/torch/include/ATen/ops/alias_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5143046Z inflating: build/lib/torch/include/ATen/ops/alias_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5143423Z inflating: build/lib/torch/include/ATen/ops/alias_copy_native.h 2024-08-08T20:48:49.5143751Z inflating: build/lib/torch/include/ATen/ops/alias_copy_ops.h 2024-08-08T20:48:49.5144307Z inflating: build/lib/torch/include/ATen/ops/alias_native.h 2024-08-08T20:48:49.5145188Z inflating: build/lib/torch/include/ATen/ops/alias_ops.h 2024-08-08T20:48:49.5146122Z inflating: build/lib/torch/include/ATen/ops/align_as.h 2024-08-08T20:48:49.5147533Z inflating: build/lib/torch/include/ATen/ops/align_as_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5148206Z inflating: build/lib/torch/include/ATen/ops/align_as_native.h 2024-08-08T20:48:49.5149171Z inflating: build/lib/torch/include/ATen/ops/align_as_ops.h 2024-08-08T20:48:49.5150113Z inflating: build/lib/torch/include/ATen/ops/align_tensors.h 2024-08-08T20:48:49.5151509Z inflating: build/lib/torch/include/ATen/ops/align_tensors_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5152158Z inflating: build/lib/torch/include/ATen/ops/align_tensors_native.h 2024-08-08T20:48:49.5153301Z inflating: build/lib/torch/include/ATen/ops/align_tensors_ops.h 2024-08-08T20:48:49.5154175Z inflating: build/lib/torch/include/ATen/ops/align_to.h 2024-08-08T20:48:49.5155410Z inflating: build/lib/torch/include/ATen/ops/align_to_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5156101Z inflating: build/lib/torch/include/ATen/ops/align_to_native.h 2024-08-08T20:48:49.5157096Z inflating: build/lib/torch/include/ATen/ops/align_to_ops.h 2024-08-08T20:48:49.5158135Z inflating: build/lib/torch/include/ATen/ops/all.h 2024-08-08T20:48:49.5159709Z inflating: build/lib/torch/include/ATen/ops/all_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5160802Z inflating: build/lib/torch/include/ATen/ops/all_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5161719Z inflating: build/lib/torch/include/ATen/ops/all_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5162566Z inflating: build/lib/torch/include/ATen/ops/all_cpu_dispatch.h 2024-08-08T20:48:49.5163537Z inflating: build/lib/torch/include/ATen/ops/all_cuda_dispatch.h 2024-08-08T20:48:49.5164416Z inflating: build/lib/torch/include/ATen/ops/all_meta.h 2024-08-08T20:48:49.5165457Z inflating: build/lib/torch/include/ATen/ops/all_meta_dispatch.h 2024-08-08T20:48:49.5166386Z inflating: build/lib/torch/include/ATen/ops/all_native.h 2024-08-08T20:48:49.5167958Z inflating: build/lib/torch/include/ATen/ops/all_ops.h 2024-08-08T20:48:49.5168438Z inflating: build/lib/torch/include/ATen/ops/allclose.h 2024-08-08T20:48:49.5169823Z inflating: build/lib/torch/include/ATen/ops/allclose_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5170478Z inflating: build/lib/torch/include/ATen/ops/allclose_native.h 2024-08-08T20:48:49.5171497Z inflating: build/lib/torch/include/ATen/ops/allclose_ops.h 2024-08-08T20:48:49.5172515Z inflating: build/lib/torch/include/ATen/ops/alpha_dropout.h 2024-08-08T20:48:49.5173752Z inflating: build/lib/torch/include/ATen/ops/alpha_dropout_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5174449Z inflating: build/lib/torch/include/ATen/ops/alpha_dropout_native.h 2024-08-08T20:48:49.5175758Z inflating: build/lib/torch/include/ATen/ops/alpha_dropout_ops.h 2024-08-08T20:48:49.5176557Z inflating: build/lib/torch/include/ATen/ops/amax.h 2024-08-08T20:48:49.5177891Z inflating: build/lib/torch/include/ATen/ops/amax_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5178525Z inflating: build/lib/torch/include/ATen/ops/amax_cpu_dispatch.h 2024-08-08T20:48:49.5179500Z inflating: build/lib/torch/include/ATen/ops/amax_cuda_dispatch.h 2024-08-08T20:48:49.5180304Z inflating: build/lib/torch/include/ATen/ops/amax_meta.h 2024-08-08T20:48:49.5181348Z inflating: build/lib/torch/include/ATen/ops/amax_meta_dispatch.h 2024-08-08T20:48:49.5182201Z inflating: build/lib/torch/include/ATen/ops/amax_native.h 2024-08-08T20:48:49.5183233Z inflating: build/lib/torch/include/ATen/ops/amax_ops.h 2024-08-08T20:48:49.5184124Z inflating: build/lib/torch/include/ATen/ops/amin.h 2024-08-08T20:48:49.5185479Z inflating: build/lib/torch/include/ATen/ops/amin_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5186151Z inflating: build/lib/torch/include/ATen/ops/amin_cpu_dispatch.h 2024-08-08T20:48:49.5187144Z inflating: build/lib/torch/include/ATen/ops/amin_cuda_dispatch.h 2024-08-08T20:48:49.5188059Z inflating: build/lib/torch/include/ATen/ops/amin_meta.h 2024-08-08T20:48:49.5189058Z inflating: build/lib/torch/include/ATen/ops/amin_meta_dispatch.h 2024-08-08T20:48:49.5190419Z inflating: build/lib/torch/include/ATen/ops/amin_native.h 2024-08-08T20:48:49.5191261Z inflating: build/lib/torch/include/ATen/ops/amin_ops.h 2024-08-08T20:48:49.5192369Z inflating: build/lib/torch/include/ATen/ops/aminmax.h 2024-08-08T20:48:49.5193702Z inflating: build/lib/torch/include/ATen/ops/aminmax_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5194322Z inflating: build/lib/torch/include/ATen/ops/aminmax_cpu_dispatch.h 2024-08-08T20:48:49.5195334Z inflating: build/lib/torch/include/ATen/ops/aminmax_cuda_dispatch.h 2024-08-08T20:48:49.5196175Z inflating: build/lib/torch/include/ATen/ops/aminmax_meta.h 2024-08-08T20:48:49.5197274Z inflating: build/lib/torch/include/ATen/ops/aminmax_meta_dispatch.h 2024-08-08T20:48:49.5198151Z inflating: build/lib/torch/include/ATen/ops/aminmax_native.h 2024-08-08T20:48:49.5199151Z inflating: build/lib/torch/include/ATen/ops/aminmax_ops.h 2024-08-08T20:48:49.5200680Z inflating: build/lib/torch/include/ATen/ops/and.h 2024-08-08T20:48:49.5201774Z inflating: build/lib/torch/include/ATen/ops/and_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5202471Z inflating: build/lib/torch/include/ATen/ops/and_native.h 2024-08-08T20:48:49.5204192Z inflating: build/lib/torch/include/ATen/ops/and_ops.h 2024-08-08T20:48:49.5204827Z inflating: build/lib/torch/include/ATen/ops/angle.h 2024-08-08T20:48:49.5206288Z inflating: build/lib/torch/include/ATen/ops/angle_cpu_dispatch.h 2024-08-08T20:48:49.5206999Z inflating: build/lib/torch/include/ATen/ops/angle_cuda_dispatch.h 2024-08-08T20:48:49.5208117Z inflating: build/lib/torch/include/ATen/ops/angle_native.h 2024-08-08T20:48:49.5209896Z inflating: build/lib/torch/include/ATen/ops/angle_ops.h 2024-08-08T20:48:49.5210825Z inflating: build/lib/torch/include/ATen/ops/any.h 2024-08-08T20:48:49.5212098Z inflating: build/lib/torch/include/ATen/ops/any_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5213329Z inflating: build/lib/torch/include/ATen/ops/any_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5214223Z inflating: build/lib/torch/include/ATen/ops/any_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5215052Z inflating: build/lib/torch/include/ATen/ops/any_cpu_dispatch.h 2024-08-08T20:48:49.5216099Z inflating: build/lib/torch/include/ATen/ops/any_cuda_dispatch.h 2024-08-08T20:48:49.5216961Z inflating: build/lib/torch/include/ATen/ops/any_meta.h 2024-08-08T20:48:49.5218090Z inflating: build/lib/torch/include/ATen/ops/any_meta_dispatch.h 2024-08-08T20:48:49.5219051Z inflating: build/lib/torch/include/ATen/ops/any_native.h 2024-08-08T20:48:49.5220494Z inflating: build/lib/torch/include/ATen/ops/any_ops.h 2024-08-08T20:48:49.5221451Z inflating: build/lib/torch/include/ATen/ops/arange.h 2024-08-08T20:48:49.5222755Z inflating: build/lib/torch/include/ATen/ops/arange_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5223562Z inflating: build/lib/torch/include/ATen/ops/arange_cpu_dispatch.h 2024-08-08T20:48:49.5224531Z inflating: build/lib/torch/include/ATen/ops/arange_cuda_dispatch.h 2024-08-08T20:48:49.5225491Z inflating: build/lib/torch/include/ATen/ops/arange_meta_dispatch.h 2024-08-08T20:48:49.5226458Z inflating: build/lib/torch/include/ATen/ops/arange_native.h 2024-08-08T20:48:49.5227882Z inflating: build/lib/torch/include/ATen/ops/arange_ops.h 2024-08-08T20:48:49.5228608Z inflating: build/lib/torch/include/ATen/ops/arccos.h 2024-08-08T20:48:49.5229915Z inflating: build/lib/torch/include/ATen/ops/arccos_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5230612Z inflating: build/lib/torch/include/ATen/ops/arccos_native.h 2024-08-08T20:48:49.5231648Z inflating: build/lib/torch/include/ATen/ops/arccos_ops.h 2024-08-08T20:48:49.5232720Z inflating: build/lib/torch/include/ATen/ops/arccosh.h 2024-08-08T20:48:49.5233962Z inflating: build/lib/torch/include/ATen/ops/arccosh_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5234672Z inflating: build/lib/torch/include/ATen/ops/arccosh_native.h 2024-08-08T20:48:49.5235707Z inflating: build/lib/torch/include/ATen/ops/arccosh_ops.h 2024-08-08T20:48:49.5236765Z inflating: build/lib/torch/include/ATen/ops/arcsin.h 2024-08-08T20:48:49.5237966Z inflating: build/lib/torch/include/ATen/ops/arcsin_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5238611Z inflating: build/lib/torch/include/ATen/ops/arcsin_native.h 2024-08-08T20:48:49.5239624Z inflating: build/lib/torch/include/ATen/ops/arcsin_ops.h 2024-08-08T20:48:49.5240613Z inflating: build/lib/torch/include/ATen/ops/arcsinh.h 2024-08-08T20:48:49.5242314Z inflating: build/lib/torch/include/ATen/ops/arcsinh_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5242765Z inflating: build/lib/torch/include/ATen/ops/arcsinh_native.h 2024-08-08T20:48:49.5243987Z inflating: build/lib/torch/include/ATen/ops/arcsinh_ops.h 2024-08-08T20:48:49.5244822Z inflating: build/lib/torch/include/ATen/ops/arctan.h 2024-08-08T20:48:49.5245839Z inflating: build/lib/torch/include/ATen/ops/arctan2.h 2024-08-08T20:48:49.5247123Z inflating: build/lib/torch/include/ATen/ops/arctan2_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5247854Z inflating: build/lib/torch/include/ATen/ops/arctan2_native.h 2024-08-08T20:48:49.5248878Z inflating: build/lib/torch/include/ATen/ops/arctan2_ops.h 2024-08-08T20:48:49.5250108Z inflating: build/lib/torch/include/ATen/ops/arctan_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5250835Z inflating: build/lib/torch/include/ATen/ops/arctan_native.h 2024-08-08T20:48:49.5251858Z inflating: build/lib/torch/include/ATen/ops/arctan_ops.h 2024-08-08T20:48:49.5252830Z inflating: build/lib/torch/include/ATen/ops/arctanh.h 2024-08-08T20:48:49.5254077Z inflating: build/lib/torch/include/ATen/ops/arctanh_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5254831Z inflating: build/lib/torch/include/ATen/ops/arctanh_native.h 2024-08-08T20:48:49.5255890Z inflating: build/lib/torch/include/ATen/ops/arctanh_ops.h 2024-08-08T20:48:49.5256819Z inflating: build/lib/torch/include/ATen/ops/argmax.h 2024-08-08T20:48:49.5258263Z inflating: build/lib/torch/include/ATen/ops/argmax_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5259148Z inflating: build/lib/torch/include/ATen/ops/argmax_cpu_dispatch.h 2024-08-08T20:48:49.5260244Z inflating: build/lib/torch/include/ATen/ops/argmax_cuda_dispatch.h 2024-08-08T20:48:49.5261183Z inflating: build/lib/torch/include/ATen/ops/argmax_meta.h 2024-08-08T20:48:49.5262243Z inflating: build/lib/torch/include/ATen/ops/argmax_meta_dispatch.h 2024-08-08T20:48:49.5263155Z inflating: build/lib/torch/include/ATen/ops/argmax_native.h 2024-08-08T20:48:49.5264161Z inflating: build/lib/torch/include/ATen/ops/argmax_ops.h 2024-08-08T20:48:49.5265121Z inflating: build/lib/torch/include/ATen/ops/argmin.h 2024-08-08T20:48:49.5266586Z inflating: build/lib/torch/include/ATen/ops/argmin_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5267237Z inflating: build/lib/torch/include/ATen/ops/argmin_cpu_dispatch.h 2024-08-08T20:48:49.5268241Z inflating: build/lib/torch/include/ATen/ops/argmin_cuda_dispatch.h 2024-08-08T20:48:49.5269084Z inflating: build/lib/torch/include/ATen/ops/argmin_meta.h 2024-08-08T20:48:49.5270140Z inflating: build/lib/torch/include/ATen/ops/argmin_meta_dispatch.h 2024-08-08T20:48:49.5271057Z inflating: build/lib/torch/include/ATen/ops/argmin_native.h 2024-08-08T20:48:49.5272054Z inflating: build/lib/torch/include/ATen/ops/argmin_ops.h 2024-08-08T20:48:49.5273075Z inflating: build/lib/torch/include/ATen/ops/argsort.h 2024-08-08T20:48:49.5274793Z inflating: build/lib/torch/include/ATen/ops/argsort_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5276252Z inflating: build/lib/torch/include/ATen/ops/argsort_native.h 2024-08-08T20:48:49.5277153Z inflating: build/lib/torch/include/ATen/ops/argsort_ops.h 2024-08-08T20:48:49.5278027Z inflating: build/lib/torch/include/ATen/ops/argwhere.h 2024-08-08T20:48:49.5279536Z inflating: build/lib/torch/include/ATen/ops/argwhere_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5280135Z inflating: build/lib/torch/include/ATen/ops/argwhere_native.h 2024-08-08T20:48:49.5281058Z inflating: build/lib/torch/include/ATen/ops/argwhere_ops.h 2024-08-08T20:48:49.5282177Z inflating: build/lib/torch/include/ATen/ops/as_strided.h 2024-08-08T20:48:49.5283605Z inflating: build/lib/torch/include/ATen/ops/as_strided_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5284564Z inflating: build/lib/torch/include/ATen/ops/as_strided_copy.h 2024-08-08T20:48:49.5285873Z inflating: build/lib/torch/include/ATen/ops/as_strided_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5287052Z inflating: build/lib/torch/include/ATen/ops/as_strided_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5287676Z inflating: build/lib/torch/include/ATen/ops/as_strided_copy_native.h 2024-08-08T20:48:49.5288736Z inflating: build/lib/torch/include/ATen/ops/as_strided_copy_ops.h 2024-08-08T20:48:49.5289805Z inflating: build/lib/torch/include/ATen/ops/as_strided_cpu_dispatch.h 2024-08-08T20:48:49.5290775Z inflating: build/lib/torch/include/ATen/ops/as_strided_cuda_dispatch.h 2024-08-08T20:48:49.5291747Z inflating: build/lib/torch/include/ATen/ops/as_strided_meta_dispatch.h 2024-08-08T20:48:49.5292650Z inflating: build/lib/torch/include/ATen/ops/as_strided_native.h 2024-08-08T20:48:49.5293696Z inflating: build/lib/torch/include/ATen/ops/as_strided_ops.h 2024-08-08T20:48:49.5295317Z inflating: build/lib/torch/include/ATen/ops/as_strided_scatter.h 2024-08-08T20:48:49.5296420Z inflating: build/lib/torch/include/ATen/ops/as_strided_scatter_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5297629Z inflating: build/lib/torch/include/ATen/ops/as_strided_scatter_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5298231Z inflating: build/lib/torch/include/ATen/ops/as_strided_scatter_native.h 2024-08-08T20:48:49.5299322Z inflating: build/lib/torch/include/ATen/ops/as_strided_scatter_ops.h 2024-08-08T20:48:49.5300145Z inflating: build/lib/torch/include/ATen/ops/asin.h 2024-08-08T20:48:49.5301589Z inflating: build/lib/torch/include/ATen/ops/asin_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5302232Z inflating: build/lib/torch/include/ATen/ops/asin_cpu_dispatch.h 2024-08-08T20:48:49.5303248Z inflating: build/lib/torch/include/ATen/ops/asin_cuda_dispatch.h 2024-08-08T20:48:49.5304136Z inflating: build/lib/torch/include/ATen/ops/asin_meta.h 2024-08-08T20:48:49.5305163Z inflating: build/lib/torch/include/ATen/ops/asin_meta_dispatch.h 2024-08-08T20:48:49.5306078Z inflating: build/lib/torch/include/ATen/ops/asin_native.h 2024-08-08T20:48:49.5307108Z inflating: build/lib/torch/include/ATen/ops/asin_ops.h 2024-08-08T20:48:49.5308287Z inflating: build/lib/torch/include/ATen/ops/asinh.h 2024-08-08T20:48:49.5309667Z inflating: build/lib/torch/include/ATen/ops/asinh_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5310311Z inflating: build/lib/torch/include/ATen/ops/asinh_cpu_dispatch.h 2024-08-08T20:48:49.5311293Z inflating: build/lib/torch/include/ATen/ops/asinh_cuda_dispatch.h 2024-08-08T20:48:49.5312136Z inflating: build/lib/torch/include/ATen/ops/asinh_meta.h 2024-08-08T20:48:49.5313176Z inflating: build/lib/torch/include/ATen/ops/asinh_meta_dispatch.h 2024-08-08T20:48:49.5314102Z inflating: build/lib/torch/include/ATen/ops/asinh_native.h 2024-08-08T20:48:49.5315161Z inflating: build/lib/torch/include/ATen/ops/asinh_ops.h 2024-08-08T20:48:49.5316092Z inflating: build/lib/torch/include/ATen/ops/atan.h 2024-08-08T20:48:49.5317083Z inflating: build/lib/torch/include/ATen/ops/atan2.h 2024-08-08T20:48:49.5318545Z inflating: build/lib/torch/include/ATen/ops/atan2_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5319345Z inflating: build/lib/torch/include/ATen/ops/atan2_cpu_dispatch.h 2024-08-08T20:48:49.5320407Z inflating: build/lib/torch/include/ATen/ops/atan2_cuda_dispatch.h 2024-08-08T20:48:49.5321233Z inflating: build/lib/torch/include/ATen/ops/atan2_meta.h 2024-08-08T20:48:49.5322238Z inflating: build/lib/torch/include/ATen/ops/atan2_meta_dispatch.h 2024-08-08T20:48:49.5323132Z inflating: build/lib/torch/include/ATen/ops/atan2_native.h 2024-08-08T20:48:49.5324192Z inflating: build/lib/torch/include/ATen/ops/atan2_ops.h 2024-08-08T20:48:49.5325589Z inflating: build/lib/torch/include/ATen/ops/atan_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5326300Z inflating: build/lib/torch/include/ATen/ops/atan_cpu_dispatch.h 2024-08-08T20:48:49.5327252Z inflating: build/lib/torch/include/ATen/ops/atan_cuda_dispatch.h 2024-08-08T20:48:49.5328126Z inflating: build/lib/torch/include/ATen/ops/atan_meta.h 2024-08-08T20:48:49.5329180Z inflating: build/lib/torch/include/ATen/ops/atan_meta_dispatch.h 2024-08-08T20:48:49.5330102Z inflating: build/lib/torch/include/ATen/ops/atan_native.h 2024-08-08T20:48:49.5331171Z inflating: build/lib/torch/include/ATen/ops/atan_ops.h 2024-08-08T20:48:49.5332136Z inflating: build/lib/torch/include/ATen/ops/atanh.h 2024-08-08T20:48:49.5333529Z inflating: build/lib/torch/include/ATen/ops/atanh_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5334268Z inflating: build/lib/torch/include/ATen/ops/atanh_cpu_dispatch.h 2024-08-08T20:48:49.5335210Z inflating: build/lib/torch/include/ATen/ops/atanh_cuda_dispatch.h 2024-08-08T20:48:49.5336112Z inflating: build/lib/torch/include/ATen/ops/atanh_meta.h 2024-08-08T20:48:49.5337140Z inflating: build/lib/torch/include/ATen/ops/atanh_meta_dispatch.h 2024-08-08T20:48:49.5338078Z inflating: build/lib/torch/include/ATen/ops/atanh_native.h 2024-08-08T20:48:49.5339130Z inflating: build/lib/torch/include/ATen/ops/atanh_ops.h 2024-08-08T20:48:49.5340569Z inflating: build/lib/torch/include/ATen/ops/atleast_1d.h 2024-08-08T20:48:49.5342493Z inflating: build/lib/torch/include/ATen/ops/atleast_1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5343064Z inflating: build/lib/torch/include/ATen/ops/atleast_1d_native.h 2024-08-08T20:48:49.5344021Z inflating: build/lib/torch/include/ATen/ops/atleast_1d_ops.h 2024-08-08T20:48:49.5344986Z inflating: build/lib/torch/include/ATen/ops/atleast_2d.h 2024-08-08T20:48:49.5346270Z inflating: build/lib/torch/include/ATen/ops/atleast_2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5346998Z inflating: build/lib/torch/include/ATen/ops/atleast_2d_native.h 2024-08-08T20:48:49.5348002Z inflating: build/lib/torch/include/ATen/ops/atleast_2d_ops.h 2024-08-08T20:48:49.5348940Z inflating: build/lib/torch/include/ATen/ops/atleast_3d.h 2024-08-08T20:48:49.5350271Z inflating: build/lib/torch/include/ATen/ops/atleast_3d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5350953Z inflating: build/lib/torch/include/ATen/ops/atleast_3d_native.h 2024-08-08T20:48:49.5351977Z inflating: build/lib/torch/include/ATen/ops/atleast_3d_ops.h 2024-08-08T20:48:49.5352898Z inflating: build/lib/torch/include/ATen/ops/avg_pool1d.h 2024-08-08T20:48:49.5354202Z inflating: build/lib/torch/include/ATen/ops/avg_pool1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5354905Z inflating: build/lib/torch/include/ATen/ops/avg_pool1d_native.h 2024-08-08T20:48:49.5355898Z inflating: build/lib/torch/include/ATen/ops/avg_pool1d_ops.h 2024-08-08T20:48:49.5356975Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d.h 2024-08-08T20:48:49.5358079Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_backward.h 2024-08-08T20:48:49.5359822Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5360542Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.5361732Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.5362632Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_backward_meta.h 2024-08-08T20:48:49.5363668Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_backward_meta_dispatch.h 2024-08-08T20:48:49.5364715Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_backward_native.h 2024-08-08T20:48:49.5365781Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_backward_ops.h 2024-08-08T20:48:49.5367160Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5367920Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_cpu_dispatch.h 2024-08-08T20:48:49.5368998Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_cuda_dispatch.h 2024-08-08T20:48:49.5370064Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_meta.h 2024-08-08T20:48:49.5371141Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_meta_dispatch.h 2024-08-08T20:48:49.5372138Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_native.h 2024-08-08T20:48:49.5373188Z inflating: build/lib/torch/include/ATen/ops/avg_pool2d_ops.h 2024-08-08T20:48:49.5374656Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d.h 2024-08-08T20:48:49.5375548Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_backward.h 2024-08-08T20:48:49.5377031Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5377787Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_backward_cpu_dispatch.h 2024-08-08T20:48:49.5378809Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_backward_cuda_dispatch.h 2024-08-08T20:48:49.5379728Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_backward_meta.h 2024-08-08T20:48:49.5380842Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_backward_meta_dispatch.h 2024-08-08T20:48:49.5381838Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_backward_native.h 2024-08-08T20:48:49.5382946Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_backward_ops.h 2024-08-08T20:48:49.5384253Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5384993Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_cpu_dispatch.h 2024-08-08T20:48:49.5386004Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_cuda_dispatch.h 2024-08-08T20:48:49.5386903Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_meta.h 2024-08-08T20:48:49.5387992Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_meta_dispatch.h 2024-08-08T20:48:49.5388951Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_native.h 2024-08-08T20:48:49.5390083Z inflating: build/lib/torch/include/ATen/ops/avg_pool3d_ops.h 2024-08-08T20:48:49.5391062Z inflating: build/lib/torch/include/ATen/ops/baddbmm.h 2024-08-08T20:48:49.5392464Z inflating: build/lib/torch/include/ATen/ops/baddbmm_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5393192Z inflating: build/lib/torch/include/ATen/ops/baddbmm_cpu_dispatch.h 2024-08-08T20:48:49.5394228Z inflating: build/lib/torch/include/ATen/ops/baddbmm_cuda_dispatch.h 2024-08-08T20:48:49.5395299Z inflating: build/lib/torch/include/ATen/ops/baddbmm_meta.h 2024-08-08T20:48:49.5396382Z inflating: build/lib/torch/include/ATen/ops/baddbmm_meta_dispatch.h 2024-08-08T20:48:49.5397329Z inflating: build/lib/torch/include/ATen/ops/baddbmm_native.h 2024-08-08T20:48:49.5398738Z inflating: build/lib/torch/include/ATen/ops/baddbmm_ops.h 2024-08-08T20:48:49.5399608Z inflating: build/lib/torch/include/ATen/ops/bartlett_window.h 2024-08-08T20:48:49.5400968Z inflating: build/lib/torch/include/ATen/ops/bartlett_window_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5401669Z inflating: build/lib/torch/include/ATen/ops/bartlett_window_native.h 2024-08-08T20:48:49.5403043Z inflating: build/lib/torch/include/ATen/ops/bartlett_window_ops.h 2024-08-08T20:48:49.5403817Z inflating: build/lib/torch/include/ATen/ops/batch_norm.h 2024-08-08T20:48:49.5404871Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward.h 2024-08-08T20:48:49.5406053Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_cpu_dispatch.h 2024-08-08T20:48:49.5407117Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_cuda_dispatch.h 2024-08-08T20:48:49.5408318Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_elemt.h 2024-08-08T20:48:49.5410481Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_elemt_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5411189Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_elemt_cuda_dispatch.h 2024-08-08T20:48:49.5412029Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_elemt_native.h 2024-08-08T20:48:49.5413174Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_elemt_ops.h 2024-08-08T20:48:49.5414119Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_native.h 2024-08-08T20:48:49.5415185Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_ops.h 2024-08-08T20:48:49.5416285Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_reduce.h 2024-08-08T20:48:49.5417634Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_reduce_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5418497Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_reduce_cuda_dispatch.h 2024-08-08T20:48:49.5419425Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_reduce_native.h 2024-08-08T20:48:49.5420561Z inflating: build/lib/torch/include/ATen/ops/batch_norm_backward_reduce_ops.h 2024-08-08T20:48:49.5421684Z inflating: build/lib/torch/include/ATen/ops/batch_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5422481Z inflating: build/lib/torch/include/ATen/ops/batch_norm_elemt.h 2024-08-08T20:48:49.5423630Z inflating: build/lib/torch/include/ATen/ops/batch_norm_elemt_cuda_dispatch.h 2024-08-08T20:48:49.5424516Z inflating: build/lib/torch/include/ATen/ops/batch_norm_elemt_native.h 2024-08-08T20:48:49.5425590Z inflating: build/lib/torch/include/ATen/ops/batch_norm_elemt_ops.h 2024-08-08T20:48:49.5426672Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats.h 2024-08-08T20:48:49.5427984Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5428817Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats_cuda_dispatch.h 2024-08-08T20:48:49.5429765Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats_native.h 2024-08-08T20:48:49.5430973Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats_ops.h 2024-08-08T20:48:49.5432066Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats_with_counts.h 2024-08-08T20:48:49.5433374Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats_with_counts_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5434210Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats_with_counts_cuda_dispatch.h 2024-08-08T20:48:49.5435139Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats_with_counts_native.h 2024-08-08T20:48:49.5436266Z inflating: build/lib/torch/include/ATen/ops/batch_norm_gather_stats_with_counts_ops.h 2024-08-08T20:48:49.5437038Z inflating: build/lib/torch/include/ATen/ops/batch_norm_native.h 2024-08-08T20:48:49.5439161Z inflating: build/lib/torch/include/ATen/ops/batch_norm_ops.h 2024-08-08T20:48:49.5439540Z inflating: build/lib/torch/include/ATen/ops/batch_norm_stats.h 2024-08-08T20:48:49.5440440Z inflating: build/lib/torch/include/ATen/ops/batch_norm_stats_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5441382Z inflating: build/lib/torch/include/ATen/ops/batch_norm_stats_cuda_dispatch.h 2024-08-08T20:48:49.5442379Z inflating: build/lib/torch/include/ATen/ops/batch_norm_stats_native.h 2024-08-08T20:48:49.5443398Z inflating: build/lib/torch/include/ATen/ops/batch_norm_stats_ops.h 2024-08-08T20:48:49.5444485Z inflating: build/lib/torch/include/ATen/ops/batch_norm_update_stats.h 2024-08-08T20:48:49.5445797Z inflating: build/lib/torch/include/ATen/ops/batch_norm_update_stats_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5446576Z inflating: build/lib/torch/include/ATen/ops/batch_norm_update_stats_cpu_dispatch.h 2024-08-08T20:48:49.5447588Z inflating: build/lib/torch/include/ATen/ops/batch_norm_update_stats_cuda_dispatch.h 2024-08-08T20:48:49.5448502Z inflating: build/lib/torch/include/ATen/ops/batch_norm_update_stats_native.h 2024-08-08T20:48:49.5449568Z inflating: build/lib/torch/include/ATen/ops/batch_norm_update_stats_ops.h 2024-08-08T20:48:49.5450576Z inflating: build/lib/torch/include/ATen/ops/bernoulli.h 2024-08-08T20:48:49.5452191Z inflating: build/lib/torch/include/ATen/ops/bernoulli_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5453062Z inflating: build/lib/torch/include/ATen/ops/bernoulli_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5453827Z inflating: build/lib/torch/include/ATen/ops/bernoulli_cpu_dispatch.h 2024-08-08T20:48:49.5454817Z inflating: build/lib/torch/include/ATen/ops/bernoulli_cuda_dispatch.h 2024-08-08T20:48:49.5455811Z inflating: build/lib/torch/include/ATen/ops/bernoulli_meta_dispatch.h 2024-08-08T20:48:49.5456763Z inflating: build/lib/torch/include/ATen/ops/bernoulli_native.h 2024-08-08T20:48:49.5458323Z inflating: build/lib/torch/include/ATen/ops/bernoulli_ops.h 2024-08-08T20:48:49.5462462Z inflating: build/lib/torch/include/ATen/ops/bilinear.h 2024-08-08T20:48:49.5463618Z inflating: build/lib/torch/include/ATen/ops/bilinear_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5464325Z inflating: build/lib/torch/include/ATen/ops/bilinear_native.h 2024-08-08T20:48:49.5465304Z inflating: build/lib/torch/include/ATen/ops/bilinear_ops.h 2024-08-08T20:48:49.5466397Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy.h 2024-08-08T20:48:49.5467542Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_backward.h 2024-08-08T20:48:49.5468760Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_backward_cpu_dispatch.h 2024-08-08T20:48:49.5470336Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_backward_cuda_dispatch.h 2024-08-08T20:48:49.5471667Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_backward_native.h 2024-08-08T20:48:49.5472819Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_backward_ops.h 2024-08-08T20:48:49.5474512Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_cpu_dispatch.h 2024-08-08T20:48:49.5475703Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_cuda_dispatch.h 2024-08-08T20:48:49.5476838Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_native.h 2024-08-08T20:48:49.5477979Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_ops.h 2024-08-08T20:48:49.5479089Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_with_logits.h 2024-08-08T20:48:49.5480518Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_with_logits_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5481974Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_with_logits_native.h 2024-08-08T20:48:49.5483199Z inflating: build/lib/torch/include/ATen/ops/binary_cross_entropy_with_logits_ops.h 2024-08-08T20:48:49.5484174Z inflating: build/lib/torch/include/ATen/ops/bincount.h 2024-08-08T20:48:49.5485248Z inflating: build/lib/torch/include/ATen/ops/bincount_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5486438Z inflating: build/lib/torch/include/ATen/ops/bincount_cpu_dispatch.h 2024-08-08T20:48:49.5487511Z inflating: build/lib/torch/include/ATen/ops/bincount_cuda_dispatch.h 2024-08-08T20:48:49.5488483Z inflating: build/lib/torch/include/ATen/ops/bincount_native.h 2024-08-08T20:48:49.5489289Z inflating: build/lib/torch/include/ATen/ops/bincount_ops.h 2024-08-08T20:48:49.5490113Z inflating: build/lib/torch/include/ATen/ops/binomial.h 2024-08-08T20:48:49.5491158Z inflating: build/lib/torch/include/ATen/ops/binomial_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5492332Z inflating: build/lib/torch/include/ATen/ops/binomial_cpu_dispatch.h 2024-08-08T20:48:49.5493326Z inflating: build/lib/torch/include/ATen/ops/binomial_cuda_dispatch.h 2024-08-08T20:48:49.5494280Z inflating: build/lib/torch/include/ATen/ops/binomial_native.h 2024-08-08T20:48:49.5495169Z inflating: build/lib/torch/include/ATen/ops/binomial_ops.h 2024-08-08T20:48:49.5495983Z inflating: build/lib/torch/include/ATen/ops/bitwise_and.h 2024-08-08T20:48:49.5497333Z inflating: build/lib/torch/include/ATen/ops/bitwise_and_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5498797Z inflating: build/lib/torch/include/ATen/ops/bitwise_and_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5500118Z inflating: build/lib/torch/include/ATen/ops/bitwise_and_cpu_dispatch.h 2024-08-08T20:48:49.5501167Z inflating: build/lib/torch/include/ATen/ops/bitwise_and_cuda_dispatch.h 2024-08-08T20:48:49.5502143Z inflating: build/lib/torch/include/ATen/ops/bitwise_and_meta.h 2024-08-08T20:48:49.5503097Z inflating: build/lib/torch/include/ATen/ops/bitwise_and_meta_dispatch.h 2024-08-08T20:48:49.5504084Z inflating: build/lib/torch/include/ATen/ops/bitwise_and_native.h 2024-08-08T20:48:49.5504969Z inflating: build/lib/torch/include/ATen/ops/bitwise_and_ops.h 2024-08-08T20:48:49.5505816Z inflating: build/lib/torch/include/ATen/ops/bitwise_left_shift.h 2024-08-08T20:48:49.5507085Z inflating: build/lib/torch/include/ATen/ops/bitwise_left_shift_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5508697Z inflating: build/lib/torch/include/ATen/ops/bitwise_left_shift_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5510121Z inflating: build/lib/torch/include/ATen/ops/bitwise_left_shift_cpu_dispatch.h 2024-08-08T20:48:49.5511249Z inflating: build/lib/torch/include/ATen/ops/bitwise_left_shift_cuda_dispatch.h 2024-08-08T20:48:49.5512335Z inflating: build/lib/torch/include/ATen/ops/bitwise_left_shift_meta.h 2024-08-08T20:48:49.5513414Z inflating: build/lib/torch/include/ATen/ops/bitwise_left_shift_meta_dispatch.h 2024-08-08T20:48:49.5514484Z inflating: build/lib/torch/include/ATen/ops/bitwise_left_shift_native.h 2024-08-08T20:48:49.5515497Z inflating: build/lib/torch/include/ATen/ops/bitwise_left_shift_ops.h 2024-08-08T20:48:49.5516499Z inflating: build/lib/torch/include/ATen/ops/bitwise_not.h 2024-08-08T20:48:49.5517760Z inflating: build/lib/torch/include/ATen/ops/bitwise_not_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5519064Z inflating: build/lib/torch/include/ATen/ops/bitwise_not_cpu_dispatch.h 2024-08-08T20:48:49.5520107Z inflating: build/lib/torch/include/ATen/ops/bitwise_not_cuda_dispatch.h 2024-08-08T20:48:49.5521082Z inflating: build/lib/torch/include/ATen/ops/bitwise_not_meta.h 2024-08-08T20:48:49.5522050Z inflating: build/lib/torch/include/ATen/ops/bitwise_not_meta_dispatch.h 2024-08-08T20:48:49.5523020Z inflating: build/lib/torch/include/ATen/ops/bitwise_not_native.h 2024-08-08T20:48:49.5523918Z inflating: build/lib/torch/include/ATen/ops/bitwise_not_ops.h 2024-08-08T20:48:49.5524705Z inflating: build/lib/torch/include/ATen/ops/bitwise_or.h 2024-08-08T20:48:49.5525787Z inflating: build/lib/torch/include/ATen/ops/bitwise_or_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5527304Z inflating: build/lib/torch/include/ATen/ops/bitwise_or_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5528692Z inflating: build/lib/torch/include/ATen/ops/bitwise_or_cpu_dispatch.h 2024-08-08T20:48:49.5529721Z inflating: build/lib/torch/include/ATen/ops/bitwise_or_cuda_dispatch.h 2024-08-08T20:48:49.5530576Z inflating: build/lib/torch/include/ATen/ops/bitwise_or_meta.h 2024-08-08T20:48:49.5531627Z inflating: build/lib/torch/include/ATen/ops/bitwise_or_meta_dispatch.h 2024-08-08T20:48:49.5532584Z inflating: build/lib/torch/include/ATen/ops/bitwise_or_native.h 2024-08-08T20:48:49.5533457Z inflating: build/lib/torch/include/ATen/ops/bitwise_or_ops.h 2024-08-08T20:48:49.5534357Z inflating: build/lib/torch/include/ATen/ops/bitwise_right_shift.h 2024-08-08T20:48:49.5535581Z inflating: build/lib/torch/include/ATen/ops/bitwise_right_shift_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5537176Z inflating: build/lib/torch/include/ATen/ops/bitwise_right_shift_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5538632Z inflating: build/lib/torch/include/ATen/ops/bitwise_right_shift_cpu_dispatch.h 2024-08-08T20:48:49.5539801Z inflating: build/lib/torch/include/ATen/ops/bitwise_right_shift_cuda_dispatch.h 2024-08-08T20:48:49.5540889Z inflating: build/lib/torch/include/ATen/ops/bitwise_right_shift_meta.h 2024-08-08T20:48:49.5541973Z inflating: build/lib/torch/include/ATen/ops/bitwise_right_shift_meta_dispatch.h 2024-08-08T20:48:49.5543083Z inflating: build/lib/torch/include/ATen/ops/bitwise_right_shift_native.h 2024-08-08T20:48:49.5544124Z inflating: build/lib/torch/include/ATen/ops/bitwise_right_shift_ops.h 2024-08-08T20:48:49.5545006Z inflating: build/lib/torch/include/ATen/ops/bitwise_xor.h 2024-08-08T20:48:49.5546115Z inflating: build/lib/torch/include/ATen/ops/bitwise_xor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5547514Z inflating: build/lib/torch/include/ATen/ops/bitwise_xor_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5548878Z inflating: build/lib/torch/include/ATen/ops/bitwise_xor_cpu_dispatch.h 2024-08-08T20:48:49.5549973Z inflating: build/lib/torch/include/ATen/ops/bitwise_xor_cuda_dispatch.h 2024-08-08T20:48:49.5550950Z inflating: build/lib/torch/include/ATen/ops/bitwise_xor_meta.h 2024-08-08T20:48:49.5551912Z inflating: build/lib/torch/include/ATen/ops/bitwise_xor_meta_dispatch.h 2024-08-08T20:48:49.5552843Z inflating: build/lib/torch/include/ATen/ops/bitwise_xor_native.h 2024-08-08T20:48:49.5553786Z inflating: build/lib/torch/include/ATen/ops/bitwise_xor_ops.h 2024-08-08T20:48:49.5554668Z inflating: build/lib/torch/include/ATen/ops/blackman_window.h 2024-08-08T20:48:49.5555833Z inflating: build/lib/torch/include/ATen/ops/blackman_window_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5557006Z inflating: build/lib/torch/include/ATen/ops/blackman_window_native.h 2024-08-08T20:48:49.5558128Z inflating: build/lib/torch/include/ATen/ops/blackman_window_ops.h 2024-08-08T20:48:49.5559143Z inflating: build/lib/torch/include/ATen/ops/block_diag.h 2024-08-08T20:48:49.5560322Z inflating: build/lib/torch/include/ATen/ops/block_diag_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5561472Z inflating: build/lib/torch/include/ATen/ops/block_diag_native.h 2024-08-08T20:48:49.5562308Z inflating: build/lib/torch/include/ATen/ops/block_diag_ops.h 2024-08-08T20:48:49.5563140Z inflating: build/lib/torch/include/ATen/ops/bmm.h 2024-08-08T20:48:49.5564215Z inflating: build/lib/torch/include/ATen/ops/bmm_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5565380Z inflating: build/lib/torch/include/ATen/ops/bmm_cpu_dispatch.h 2024-08-08T20:48:49.5566291Z inflating: build/lib/torch/include/ATen/ops/bmm_cuda_dispatch.h 2024-08-08T20:48:49.5567135Z inflating: build/lib/torch/include/ATen/ops/bmm_meta.h 2024-08-08T20:48:49.5567977Z inflating: build/lib/torch/include/ATen/ops/bmm_meta_dispatch.h 2024-08-08T20:48:49.5568826Z inflating: build/lib/torch/include/ATen/ops/bmm_native.h 2024-08-08T20:48:49.5569721Z inflating: build/lib/torch/include/ATen/ops/bmm_ops.h 2024-08-08T20:48:49.5570567Z inflating: build/lib/torch/include/ATen/ops/broadcast_tensors.h 2024-08-08T20:48:49.5571780Z inflating: build/lib/torch/include/ATen/ops/broadcast_tensors_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5573062Z inflating: build/lib/torch/include/ATen/ops/broadcast_tensors_native.h 2024-08-08T20:48:49.5573960Z inflating: build/lib/torch/include/ATen/ops/cummin_ops.h 2024-08-08T20:48:49.5574836Z inflating: build/lib/torch/include/ATen/ops/broadcast_tensors_ops.h 2024-08-08T20:48:49.5575674Z inflating: build/lib/torch/include/ATen/ops/broadcast_to.h 2024-08-08T20:48:49.5576823Z inflating: build/lib/torch/include/ATen/ops/broadcast_to_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5578032Z inflating: build/lib/torch/include/ATen/ops/broadcast_to_native.h 2024-08-08T20:48:49.5578929Z inflating: build/lib/torch/include/ATen/ops/broadcast_to_ops.h 2024-08-08T20:48:49.5579732Z inflating: build/lib/torch/include/ATen/ops/bucketize.h 2024-08-08T20:48:49.5580815Z inflating: build/lib/torch/include/ATen/ops/bucketize_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5581965Z inflating: build/lib/torch/include/ATen/ops/bucketize_cpu_dispatch.h 2024-08-08T20:48:49.5582967Z inflating: build/lib/torch/include/ATen/ops/bucketize_cuda_dispatch.h 2024-08-08T20:48:49.5584011Z inflating: build/lib/torch/include/ATen/ops/bucketize_native.h 2024-08-08T20:48:49.5584875Z inflating: build/lib/torch/include/ATen/ops/bucketize_ops.h 2024-08-08T20:48:49.5585684Z inflating: build/lib/torch/include/ATen/ops/can_cast.h 2024-08-08T20:48:49.5586728Z inflating: build/lib/torch/include/ATen/ops/can_cast_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5587811Z inflating: build/lib/torch/include/ATen/ops/can_cast_native.h 2024-08-08T20:48:49.5588652Z inflating: build/lib/torch/include/ATen/ops/can_cast_ops.h 2024-08-08T20:48:49.5589557Z inflating: build/lib/torch/include/ATen/ops/cartesian_prod.h 2024-08-08T20:48:49.5590710Z inflating: build/lib/torch/include/ATen/ops/cartesian_prod_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5591905Z inflating: build/lib/torch/include/ATen/ops/cartesian_prod_native.h 2024-08-08T20:48:49.5592895Z inflating: build/lib/torch/include/ATen/ops/cartesian_prod_ops.h 2024-08-08T20:48:49.5593717Z inflating: build/lib/torch/include/ATen/ops/cat.h 2024-08-08T20:48:49.5594798Z inflating: build/lib/torch/include/ATen/ops/cat_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5596133Z inflating: build/lib/torch/include/ATen/ops/cat_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5597202Z inflating: build/lib/torch/include/ATen/ops/cat_cpu_dispatch.h 2024-08-08T20:48:49.5598219Z inflating: build/lib/torch/include/ATen/ops/cat_cuda_dispatch.h 2024-08-08T20:48:49.5599111Z inflating: build/lib/torch/include/ATen/ops/cat_meta.h 2024-08-08T20:48:49.5599951Z inflating: build/lib/torch/include/ATen/ops/cat_meta_dispatch.h 2024-08-08T20:48:49.5600803Z inflating: build/lib/torch/include/ATen/ops/cat_native.h 2024-08-08T20:48:49.5601531Z inflating: build/lib/torch/include/ATen/ops/cat_ops.h 2024-08-08T20:48:49.5602301Z inflating: build/lib/torch/include/ATen/ops/cauchy.h 2024-08-08T20:48:49.5603300Z inflating: build/lib/torch/include/ATen/ops/cauchy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5604419Z inflating: build/lib/torch/include/ATen/ops/cauchy_cpu_dispatch.h 2024-08-08T20:48:49.5605344Z inflating: build/lib/torch/include/ATen/ops/cauchy_cuda_dispatch.h 2024-08-08T20:48:49.5606350Z inflating: build/lib/torch/include/ATen/ops/cauchy_meta_dispatch.h 2024-08-08T20:48:49.5607250Z inflating: build/lib/torch/include/ATen/ops/cauchy_native.h 2024-08-08T20:48:49.5608292Z inflating: build/lib/torch/include/ATen/ops/cauchy_ops.h 2024-08-08T20:48:49.5609379Z inflating: build/lib/torch/include/ATen/ops/ccol_indices.h 2024-08-08T20:48:49.5610659Z inflating: build/lib/torch/include/ATen/ops/ccol_indices_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5611842Z inflating: build/lib/torch/include/ATen/ops/ccol_indices_copy.h 2024-08-08T20:48:49.5613039Z inflating: build/lib/torch/include/ATen/ops/ccol_indices_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5614602Z inflating: build/lib/torch/include/ATen/ops/ccol_indices_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5615953Z inflating: build/lib/torch/include/ATen/ops/ccol_indices_copy_native.h 2024-08-08T20:48:49.5616956Z inflating: build/lib/torch/include/ATen/ops/ccol_indices_copy_ops.h 2024-08-08T20:48:49.5617904Z inflating: build/lib/torch/include/ATen/ops/ccol_indices_native.h 2024-08-08T20:48:49.5618811Z inflating: build/lib/torch/include/ATen/ops/ccol_indices_ops.h 2024-08-08T20:48:49.5619621Z inflating: build/lib/torch/include/ATen/ops/cdist.h 2024-08-08T20:48:49.5620637Z inflating: build/lib/torch/include/ATen/ops/cdist_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5621691Z inflating: build/lib/torch/include/ATen/ops/cdist_native.h 2024-08-08T20:48:49.5622491Z inflating: build/lib/torch/include/ATen/ops/cdist_ops.h 2024-08-08T20:48:49.5623235Z inflating: build/lib/torch/include/ATen/ops/ceil.h 2024-08-08T20:48:49.5624343Z inflating: build/lib/torch/include/ATen/ops/ceil_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5625539Z inflating: build/lib/torch/include/ATen/ops/ceil_cpu_dispatch.h 2024-08-08T20:48:49.5626461Z inflating: build/lib/torch/include/ATen/ops/ceil_cuda_dispatch.h 2024-08-08T20:48:49.5627310Z inflating: build/lib/torch/include/ATen/ops/ceil_meta.h 2024-08-08T20:48:49.5628139Z inflating: build/lib/torch/include/ATen/ops/ceil_meta_dispatch.h 2024-08-08T20:48:49.5629009Z inflating: build/lib/torch/include/ATen/ops/ceil_native.h 2024-08-08T20:48:49.5629794Z inflating: build/lib/torch/include/ATen/ops/ceil_ops.h 2024-08-08T20:48:49.5630919Z inflating: build/lib/torch/include/ATen/ops/celu.h 2024-08-08T20:48:49.5631916Z inflating: build/lib/torch/include/ATen/ops/celu_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5632981Z inflating: build/lib/torch/include/ATen/ops/celu_native.h 2024-08-08T20:48:49.5633815Z inflating: build/lib/torch/include/ATen/ops/celu_ops.h 2024-08-08T20:48:49.5634608Z inflating: build/lib/torch/include/ATen/ops/chain_matmul.h 2024-08-08T20:48:49.5635861Z inflating: build/lib/torch/include/ATen/ops/chain_matmul_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5636970Z inflating: build/lib/torch/include/ATen/ops/chain_matmul_native.h 2024-08-08T20:48:49.5637967Z inflating: build/lib/torch/include/ATen/ops/chain_matmul_ops.h 2024-08-08T20:48:49.5638868Z inflating: build/lib/torch/include/ATen/ops/chalf.h 2024-08-08T20:48:49.5639937Z inflating: build/lib/torch/include/ATen/ops/chalf_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5640994Z inflating: build/lib/torch/include/ATen/ops/chalf_native.h 2024-08-08T20:48:49.5641773Z inflating: build/lib/torch/include/ATen/ops/chalf_ops.h 2024-08-08T20:48:49.5642643Z inflating: build/lib/torch/include/ATen/ops/channel_shuffle.h 2024-08-08T20:48:49.5643789Z inflating: build/lib/torch/include/ATen/ops/channel_shuffle_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5645035Z inflating: build/lib/torch/include/ATen/ops/channel_shuffle_cpu_dispatch.h 2024-08-08T20:48:49.5646168Z inflating: build/lib/torch/include/ATen/ops/channel_shuffle_cuda_dispatch.h 2024-08-08T20:48:49.5647194Z inflating: build/lib/torch/include/ATen/ops/channel_shuffle_native.h 2024-08-08T20:48:49.5648159Z inflating: build/lib/torch/include/ATen/ops/channel_shuffle_ops.h 2024-08-08T20:48:49.5649009Z inflating: build/lib/torch/include/ATen/ops/cholesky.h 2024-08-08T20:48:49.5649888Z inflating: build/lib/torch/include/ATen/ops/cholesky_cpu_dispatch.h 2024-08-08T20:48:49.5650958Z inflating: build/lib/torch/include/ATen/ops/cholesky_cuda_dispatch.h 2024-08-08T20:48:49.5651921Z inflating: build/lib/torch/include/ATen/ops/cholesky_inverse.h 2024-08-08T20:48:49.5652898Z inflating: build/lib/torch/include/ATen/ops/cholesky_inverse_cpu_dispatch.h 2024-08-08T20:48:49.5654035Z inflating: build/lib/torch/include/ATen/ops/cholesky_inverse_cuda_dispatch.h 2024-08-08T20:48:49.5655049Z inflating: build/lib/torch/include/ATen/ops/cholesky_inverse_native.h 2024-08-08T20:48:49.5656062Z inflating: build/lib/torch/include/ATen/ops/cholesky_inverse_ops.h 2024-08-08T20:48:49.5656975Z inflating: build/lib/torch/include/ATen/ops/cholesky_native.h 2024-08-08T20:48:49.5657751Z inflating: build/lib/torch/include/ATen/ops/cholesky_ops.h 2024-08-08T20:48:49.5658901Z inflating: build/lib/torch/include/ATen/ops/cholesky_solve.h 2024-08-08T20:48:49.5660057Z inflating: build/lib/torch/include/ATen/ops/cholesky_solve_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5661269Z inflating: build/lib/torch/include/ATen/ops/cholesky_solve_native.h 2024-08-08T20:48:49.5662225Z inflating: build/lib/torch/include/ATen/ops/cholesky_solve_ops.h 2024-08-08T20:48:49.5663196Z inflating: build/lib/torch/include/ATen/ops/choose_qparams_optimized.h 2024-08-08T20:48:49.5664513Z inflating: build/lib/torch/include/ATen/ops/choose_qparams_optimized_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5665865Z inflating: build/lib/torch/include/ATen/ops/choose_qparams_optimized_native.h 2024-08-08T20:48:49.5666948Z inflating: build/lib/torch/include/ATen/ops/choose_qparams_optimized_ops.h 2024-08-08T20:48:49.5667861Z inflating: build/lib/torch/include/ATen/ops/chunk.h 2024-08-08T20:48:49.5668873Z inflating: build/lib/torch/include/ATen/ops/chunk_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5669933Z inflating: build/lib/torch/include/ATen/ops/chunk_native.h 2024-08-08T20:48:49.5670720Z inflating: build/lib/torch/include/ATen/ops/chunk_ops.h 2024-08-08T20:48:49.5671486Z inflating: build/lib/torch/include/ATen/ops/clamp.h 2024-08-08T20:48:49.5672605Z inflating: build/lib/torch/include/ATen/ops/clamp_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5673807Z inflating: build/lib/torch/include/ATen/ops/clamp_cpu_dispatch.h 2024-08-08T20:48:49.5674735Z inflating: build/lib/torch/include/ATen/ops/clamp_cuda_dispatch.h 2024-08-08T20:48:49.5675562Z inflating: build/lib/torch/include/ATen/ops/clamp_max.h 2024-08-08T20:48:49.5676765Z inflating: build/lib/torch/include/ATen/ops/clamp_max_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5678030Z inflating: build/lib/torch/include/ATen/ops/clamp_max_cpu_dispatch.h 2024-08-08T20:48:49.5679030Z inflating: build/lib/torch/include/ATen/ops/clamp_max_cuda_dispatch.h 2024-08-08T20:48:49.5680105Z inflating: build/lib/torch/include/ATen/ops/clamp_max_meta.h 2024-08-08T20:48:49.5680955Z inflating: build/lib/torch/include/ATen/ops/clamp_max_meta_dispatch.h 2024-08-08T20:48:49.5681935Z inflating: build/lib/torch/include/ATen/ops/clamp_max_native.h 2024-08-08T20:48:49.5682843Z inflating: build/lib/torch/include/ATen/ops/clamp_max_ops.h 2024-08-08T20:48:49.5683921Z inflating: build/lib/torch/include/ATen/ops/clamp_meta.h 2024-08-08T20:48:49.5684939Z inflating: build/lib/torch/include/ATen/ops/clamp_meta_dispatch.h 2024-08-08T20:48:49.5686003Z inflating: build/lib/torch/include/ATen/ops/clamp_min.h 2024-08-08T20:48:49.5687256Z inflating: build/lib/torch/include/ATen/ops/clamp_min_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5688595Z inflating: build/lib/torch/include/ATen/ops/clamp_min_cpu_dispatch.h 2024-08-08T20:48:49.5689587Z inflating: build/lib/torch/include/ATen/ops/clamp_min_cuda_dispatch.h 2024-08-08T20:48:49.5690508Z inflating: build/lib/torch/include/ATen/ops/clamp_min_meta.h 2024-08-08T20:48:49.5691481Z inflating: build/lib/torch/include/ATen/ops/clamp_min_meta_dispatch.h 2024-08-08T20:48:49.5692526Z inflating: build/lib/torch/include/ATen/ops/clamp_min_native.h 2024-08-08T20:48:49.5693418Z inflating: build/lib/torch/include/ATen/ops/clamp_min_ops.h 2024-08-08T20:48:49.5694199Z inflating: build/lib/torch/include/ATen/ops/clamp_native.h 2024-08-08T20:48:49.5695208Z inflating: build/lib/torch/include/ATen/ops/clamp_ops.h 2024-08-08T20:48:49.5696180Z inflating: build/lib/torch/include/ATen/ops/clip.h 2024-08-08T20:48:49.5697362Z inflating: build/lib/torch/include/ATen/ops/clip_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5698399Z inflating: build/lib/torch/include/ATen/ops/clip_native.h 2024-08-08T20:48:49.5699315Z inflating: build/lib/torch/include/ATen/ops/clip_ops.h 2024-08-08T20:48:49.5700255Z inflating: build/lib/torch/include/ATen/ops/clone.h 2024-08-08T20:48:49.5701545Z inflating: build/lib/torch/include/ATen/ops/clone_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5702616Z inflating: build/lib/torch/include/ATen/ops/clone_native.h 2024-08-08T20:48:49.5703559Z inflating: build/lib/torch/include/ATen/ops/clone_ops.h 2024-08-08T20:48:49.5704383Z inflating: build/lib/torch/include/ATen/ops/coalesce.h 2024-08-08T20:48:49.5705623Z inflating: build/lib/torch/include/ATen/ops/coalesce_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5706721Z inflating: build/lib/torch/include/ATen/ops/coalesce_native.h 2024-08-08T20:48:49.5707563Z inflating: build/lib/torch/include/ATen/ops/coalesce_ops.h 2024-08-08T20:48:49.5708578Z inflating: build/lib/torch/include/ATen/ops/col2im.h 2024-08-08T20:48:49.5709724Z inflating: build/lib/torch/include/ATen/ops/col2im_cpu_dispatch.h 2024-08-08T20:48:49.5710795Z inflating: build/lib/torch/include/ATen/ops/col2im_cuda_dispatch.h 2024-08-08T20:48:49.5711910Z inflating: build/lib/torch/include/ATen/ops/col2im_native.h 2024-08-08T20:48:49.5712898Z inflating: build/lib/torch/include/ATen/ops/col2im_ops.h 2024-08-08T20:48:49.5713869Z inflating: build/lib/torch/include/ATen/ops/col_indices.h 2024-08-08T20:48:49.5715086Z inflating: build/lib/torch/include/ATen/ops/col_indices_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5716222Z inflating: build/lib/torch/include/ATen/ops/col_indices_copy.h 2024-08-08T20:48:49.5717394Z inflating: build/lib/torch/include/ATen/ops/col_indices_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5718938Z inflating: build/lib/torch/include/ATen/ops/col_indices_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5720264Z inflating: build/lib/torch/include/ATen/ops/col_indices_copy_native.h 2024-08-08T20:48:49.5721241Z inflating: build/lib/torch/include/ATen/ops/col_indices_copy_ops.h 2024-08-08T20:48:49.5722181Z inflating: build/lib/torch/include/ATen/ops/col_indices_native.h 2024-08-08T20:48:49.5723206Z inflating: build/lib/torch/include/ATen/ops/col_indices_ops.h 2024-08-08T20:48:49.5724051Z inflating: build/lib/torch/include/ATen/ops/column_stack.h 2024-08-08T20:48:49.5725164Z inflating: build/lib/torch/include/ATen/ops/column_stack_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5726332Z inflating: build/lib/torch/include/ATen/ops/column_stack_native.h 2024-08-08T20:48:49.5727227Z inflating: build/lib/torch/include/ATen/ops/column_stack_ops.h 2024-08-08T20:48:49.5728086Z inflating: build/lib/torch/include/ATen/ops/combinations.h 2024-08-08T20:48:49.5729211Z inflating: build/lib/torch/include/ATen/ops/combinations_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5730375Z inflating: build/lib/torch/include/ATen/ops/combinations_native.h 2024-08-08T20:48:49.5731294Z inflating: build/lib/torch/include/ATen/ops/combinations_ops.h 2024-08-08T20:48:49.5732124Z inflating: build/lib/torch/include/ATen/ops/complex.h 2024-08-08T20:48:49.5733153Z inflating: build/lib/torch/include/ATen/ops/complex_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5734391Z inflating: build/lib/torch/include/ATen/ops/complex_cpu_dispatch.h 2024-08-08T20:48:49.5735376Z inflating: build/lib/torch/include/ATen/ops/complex_cuda_dispatch.h 2024-08-08T20:48:49.5736293Z inflating: build/lib/torch/include/ATen/ops/complex_native.h 2024-08-08T20:48:49.5737110Z inflating: build/lib/torch/include/ATen/ops/complex_ops.h 2024-08-08T20:48:49.5737889Z inflating: build/lib/torch/include/ATen/ops/concat.h 2024-08-08T20:48:49.5738912Z inflating: build/lib/torch/include/ATen/ops/concat_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5739989Z inflating: build/lib/torch/include/ATen/ops/concat_native.h 2024-08-08T20:48:49.5740811Z inflating: build/lib/torch/include/ATen/ops/concat_ops.h 2024-08-08T20:48:49.5741606Z inflating: build/lib/torch/include/ATen/ops/concatenate.h 2024-08-08T20:48:49.5742722Z inflating: build/lib/torch/include/ATen/ops/concatenate_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5743874Z inflating: build/lib/torch/include/ATen/ops/concatenate_native.h 2024-08-08T20:48:49.5744784Z inflating: build/lib/torch/include/ATen/ops/concatenate_ops.h 2024-08-08T20:48:49.5745581Z inflating: build/lib/torch/include/ATen/ops/conj.h 2024-08-08T20:48:49.5746573Z inflating: build/lib/torch/include/ATen/ops/conj_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5747597Z inflating: build/lib/torch/include/ATen/ops/conj_native.h 2024-08-08T20:48:49.5748551Z inflating: build/lib/torch/include/ATen/ops/conj_ops.h 2024-08-08T20:48:49.5749524Z inflating: build/lib/torch/include/ATen/ops/conj_physical.h 2024-08-08T20:48:49.5750815Z inflating: build/lib/torch/include/ATen/ops/conj_physical_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5752218Z inflating: build/lib/torch/include/ATen/ops/conj_physical_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5753459Z inflating: build/lib/torch/include/ATen/ops/conj_physical_cpu_dispatch.h 2024-08-08T20:48:49.5754517Z inflating: build/lib/torch/include/ATen/ops/conj_physical_cuda_dispatch.h 2024-08-08T20:48:49.5755515Z inflating: build/lib/torch/include/ATen/ops/conj_physical_native.h 2024-08-08T20:48:49.5756454Z inflating: build/lib/torch/include/ATen/ops/conj_physical_ops.h 2024-08-08T20:48:49.5757351Z inflating: build/lib/torch/include/ATen/ops/constant_pad_nd.h 2024-08-08T20:48:49.5758843Z inflating: build/lib/torch/include/ATen/ops/constant_pad_nd_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5760065Z inflating: build/lib/torch/include/ATen/ops/constant_pad_nd_native.h 2024-08-08T20:48:49.5761025Z inflating: build/lib/torch/include/ATen/ops/constant_pad_nd_ops.h 2024-08-08T20:48:49.5761899Z inflating: build/lib/torch/include/ATen/ops/contiguous.h 2024-08-08T20:48:49.5762968Z inflating: build/lib/torch/include/ATen/ops/contiguous_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5764308Z inflating: build/lib/torch/include/ATen/ops/contiguous_native.h 2024-08-08T20:48:49.5765202Z inflating: build/lib/torch/include/ATen/ops/contiguous_ops.h 2024-08-08T20:48:49.5766012Z inflating: build/lib/torch/include/ATen/ops/conv1d.h 2024-08-08T20:48:49.5767019Z inflating: build/lib/torch/include/ATen/ops/conv1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5768097Z inflating: build/lib/torch/include/ATen/ops/conv1d_native.h 2024-08-08T20:48:49.5768919Z inflating: build/lib/torch/include/ATen/ops/conv1d_ops.h 2024-08-08T20:48:49.5769664Z inflating: build/lib/torch/include/ATen/ops/conv2d.h 2024-08-08T20:48:49.5770860Z inflating: build/lib/torch/include/ATen/ops/conv2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5771941Z inflating: build/lib/torch/include/ATen/ops/conv2d_native.h 2024-08-08T20:48:49.5772753Z inflating: build/lib/torch/include/ATen/ops/conv2d_ops.h 2024-08-08T20:48:49.5773696Z inflating: build/lib/torch/include/ATen/ops/conv3d.h 2024-08-08T20:48:49.5775004Z inflating: build/lib/torch/include/ATen/ops/conv3d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5776239Z inflating: build/lib/torch/include/ATen/ops/conv3d_native.h 2024-08-08T20:48:49.5777065Z inflating: build/lib/torch/include/ATen/ops/conv3d_ops.h 2024-08-08T20:48:49.5778170Z inflating: build/lib/torch/include/ATen/ops/conv_depthwise3d.h 2024-08-08T20:48:49.5779476Z inflating: build/lib/torch/include/ATen/ops/conv_depthwise3d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5780761Z inflating: build/lib/torch/include/ATen/ops/conv_depthwise3d_cuda_dispatch.h 2024-08-08T20:48:49.5781814Z inflating: build/lib/torch/include/ATen/ops/conv_depthwise3d_native.h 2024-08-08T20:48:49.5782795Z inflating: build/lib/torch/include/ATen/ops/conv_depthwise3d_ops.h 2024-08-08T20:48:49.5783645Z inflating: build/lib/torch/include/ATen/ops/conv_tbc.h 2024-08-08T20:48:49.5784622Z inflating: build/lib/torch/include/ATen/ops/conv_tbc_backward.h 2024-08-08T20:48:49.5785949Z inflating: build/lib/torch/include/ATen/ops/conv_tbc_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5787262Z inflating: build/lib/torch/include/ATen/ops/conv_tbc_backward_native.h 2024-08-08T20:48:49.5788198Z inflating: build/lib/torch/include/ATen/ops/conv_tbc_backward_ops.h 2024-08-08T20:48:49.5789435Z inflating: build/lib/torch/include/ATen/ops/conv_tbc_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5790547Z inflating: build/lib/torch/include/ATen/ops/conv_tbc_native.h 2024-08-08T20:48:49.5791355Z inflating: build/lib/torch/include/ATen/ops/conv_tbc_ops.h 2024-08-08T20:48:49.5792196Z inflating: build/lib/torch/include/ATen/ops/conv_transpose1d.h 2024-08-08T20:48:49.5793379Z inflating: build/lib/torch/include/ATen/ops/conv_transpose1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5794621Z inflating: build/lib/torch/include/ATen/ops/conv_transpose1d_native.h 2024-08-08T20:48:49.5795616Z inflating: build/lib/torch/include/ATen/ops/conv_transpose1d_ops.h 2024-08-08T20:48:49.5796483Z inflating: build/lib/torch/include/ATen/ops/conv_transpose2d.h 2024-08-08T20:48:49.5797694Z inflating: build/lib/torch/include/ATen/ops/conv_transpose2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5798919Z inflating: build/lib/torch/include/ATen/ops/conv_transpose2d_native.h 2024-08-08T20:48:49.5799882Z inflating: build/lib/torch/include/ATen/ops/conv_transpose2d_ops.h 2024-08-08T20:48:49.5800790Z inflating: build/lib/torch/include/ATen/ops/conv_transpose3d.h 2024-08-08T20:48:49.5801971Z inflating: build/lib/torch/include/ATen/ops/conv_transpose3d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5803193Z inflating: build/lib/torch/include/ATen/ops/conv_transpose3d_native.h 2024-08-08T20:48:49.5804122Z inflating: build/lib/torch/include/ATen/ops/conv_transpose3d_ops.h 2024-08-08T20:48:49.5805046Z inflating: build/lib/torch/include/ATen/ops/convolution.h 2024-08-08T20:48:49.5806230Z inflating: build/lib/torch/include/ATen/ops/convolution_backward.h 2024-08-08T20:48:49.5807710Z inflating: build/lib/torch/include/ATen/ops/convolution_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5808976Z inflating: build/lib/torch/include/ATen/ops/convolution_backward_cuda_dispatch.h 2024-08-08T20:48:49.5810119Z inflating: build/lib/torch/include/ATen/ops/convolution_backward_native.h 2024-08-08T20:48:49.5811166Z inflating: build/lib/torch/include/ATen/ops/convolution_backward_ops.h 2024-08-08T20:48:49.5812350Z inflating: build/lib/torch/include/ATen/ops/convolution_backward_overrideable.h 2024-08-08T20:48:49.5813730Z inflating: build/lib/torch/include/ATen/ops/convolution_backward_overrideable_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5815276Z inflating: build/lib/torch/include/ATen/ops/convolution_backward_overrideable_native.h 2024-08-08T20:48:49.5816521Z inflating: build/lib/torch/include/ATen/ops/convolution_backward_overrideable_ops.h 2024-08-08T20:48:49.5817851Z inflating: build/lib/torch/include/ATen/ops/convolution_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5819146Z inflating: build/lib/torch/include/ATen/ops/convolution_native.h 2024-08-08T20:48:49.5819994Z inflating: build/lib/torch/include/ATen/ops/convolution_ops.h 2024-08-08T20:48:49.5821036Z inflating: build/lib/torch/include/ATen/ops/convolution_overrideable.h 2024-08-08T20:48:49.5822342Z inflating: build/lib/torch/include/ATen/ops/convolution_overrideable_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5823670Z inflating: build/lib/torch/include/ATen/ops/convolution_overrideable_native.h 2024-08-08T20:48:49.5824833Z inflating: build/lib/torch/include/ATen/ops/convolution_overrideable_ops.h 2024-08-08T20:48:49.5825735Z inflating: build/lib/torch/include/ATen/ops/copy.h 2024-08-08T20:48:49.5826659Z inflating: build/lib/torch/include/ATen/ops/copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5828108Z inflating: build/lib/torch/include/ATen/ops/copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5829326Z inflating: build/lib/torch/include/ATen/ops/copy_meta_dispatch.h 2024-08-08T20:48:49.5830192Z inflating: build/lib/torch/include/ATen/ops/copy_native.h 2024-08-08T20:48:49.5830955Z inflating: build/lib/torch/include/ATen/ops/copy_ops.h 2024-08-08T20:48:49.5831799Z inflating: build/lib/torch/include/ATen/ops/copy_sparse_to_sparse.h 2024-08-08T20:48:49.5833059Z inflating: build/lib/torch/include/ATen/ops/copy_sparse_to_sparse_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5834391Z inflating: build/lib/torch/include/ATen/ops/copy_sparse_to_sparse_meta_dispatch.h 2024-08-08T20:48:49.5835530Z inflating: build/lib/torch/include/ATen/ops/copy_sparse_to_sparse_native.h 2024-08-08T20:48:49.5836584Z inflating: build/lib/torch/include/ATen/ops/copy_sparse_to_sparse_ops.h 2024-08-08T20:48:49.5837494Z inflating: build/lib/torch/include/ATen/ops/copysign.h 2024-08-08T20:48:49.5838533Z inflating: build/lib/torch/include/ATen/ops/copysign_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5840008Z inflating: build/lib/torch/include/ATen/ops/copysign_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5841294Z inflating: build/lib/torch/include/ATen/ops/copysign_cpu_dispatch.h 2024-08-08T20:48:49.5842261Z inflating: build/lib/torch/include/ATen/ops/copysign_cuda_dispatch.h 2024-08-08T20:48:49.5843186Z inflating: build/lib/torch/include/ATen/ops/copysign_meta.h 2024-08-08T20:48:49.5844101Z inflating: build/lib/torch/include/ATen/ops/copysign_meta_dispatch.h 2024-08-08T20:48:49.5845009Z inflating: build/lib/torch/include/ATen/ops/copysign_native.h 2024-08-08T20:48:49.5845867Z inflating: build/lib/torch/include/ATen/ops/copysign_ops.h 2024-08-08T20:48:49.5846668Z inflating: build/lib/torch/include/ATen/ops/corrcoef.h 2024-08-08T20:48:49.5847830Z inflating: build/lib/torch/include/ATen/ops/corrcoef_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5848868Z inflating: build/lib/torch/include/ATen/ops/corrcoef_native.h 2024-08-08T20:48:49.5849764Z inflating: build/lib/torch/include/ATen/ops/corrcoef_ops.h 2024-08-08T20:48:49.5850522Z inflating: build/lib/torch/include/ATen/ops/cos.h 2024-08-08T20:48:49.5851614Z inflating: build/lib/torch/include/ATen/ops/cos_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5852785Z inflating: build/lib/torch/include/ATen/ops/cos_cpu_dispatch.h 2024-08-08T20:48:49.5853683Z inflating: build/lib/torch/include/ATen/ops/cos_cuda_dispatch.h 2024-08-08T20:48:49.5854521Z inflating: build/lib/torch/include/ATen/ops/cos_meta.h 2024-08-08T20:48:49.5855318Z inflating: build/lib/torch/include/ATen/ops/cos_meta_dispatch.h 2024-08-08T20:48:49.5856156Z inflating: build/lib/torch/include/ATen/ops/cos_native.h 2024-08-08T20:48:49.5856913Z inflating: build/lib/torch/include/ATen/ops/cos_ops.h 2024-08-08T20:48:49.5857649Z inflating: build/lib/torch/include/ATen/ops/cosh.h 2024-08-08T20:48:49.5859090Z inflating: build/lib/torch/include/ATen/ops/cosh_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5860369Z inflating: build/lib/torch/include/ATen/ops/cosh_cpu_dispatch.h 2024-08-08T20:48:49.5861287Z inflating: build/lib/torch/include/ATen/ops/cosh_cuda_dispatch.h 2024-08-08T20:48:49.5862106Z inflating: build/lib/torch/include/ATen/ops/cosh_meta.h 2024-08-08T20:48:49.5862956Z inflating: build/lib/torch/include/ATen/ops/cosh_meta_dispatch.h 2024-08-08T20:48:49.5863813Z inflating: build/lib/torch/include/ATen/ops/cosh_native.h 2024-08-08T20:48:49.5864763Z inflating: build/lib/torch/include/ATen/ops/cosh_ops.h 2024-08-08T20:48:49.5865772Z inflating: build/lib/torch/include/ATen/ops/cosine_embedding_loss.h 2024-08-08T20:48:49.5867154Z inflating: build/lib/torch/include/ATen/ops/cosine_embedding_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5868438Z inflating: build/lib/torch/include/ATen/ops/cosine_embedding_loss_native.h 2024-08-08T20:48:49.5869515Z inflating: build/lib/torch/include/ATen/ops/cosine_embedding_loss_ops.h 2024-08-08T20:48:49.5870451Z inflating: build/lib/torch/include/ATen/ops/cosine_similarity.h 2024-08-08T20:48:49.5871687Z inflating: build/lib/torch/include/ATen/ops/cosine_similarity_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5872923Z inflating: build/lib/torch/include/ATen/ops/cosine_similarity_native.h 2024-08-08T20:48:49.5873917Z inflating: build/lib/torch/include/ATen/ops/cosine_similarity_ops.h 2024-08-08T20:48:49.5874828Z inflating: build/lib/torch/include/ATen/ops/count_nonzero.h 2024-08-08T20:48:49.5875936Z inflating: build/lib/torch/include/ATen/ops/count_nonzero_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5877157Z inflating: build/lib/torch/include/ATen/ops/count_nonzero_cpu_dispatch.h 2024-08-08T20:48:49.5878214Z inflating: build/lib/torch/include/ATen/ops/count_nonzero_cuda_dispatch.h 2024-08-08T20:48:49.5879208Z inflating: build/lib/torch/include/ATen/ops/count_nonzero_native.h 2024-08-08T20:48:49.5880143Z inflating: build/lib/torch/include/ATen/ops/count_nonzero_ops.h 2024-08-08T20:48:49.5880953Z inflating: build/lib/torch/include/ATen/ops/cov.h 2024-08-08T20:48:49.5881912Z inflating: build/lib/torch/include/ATen/ops/cov_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5882859Z inflating: build/lib/torch/include/ATen/ops/cov_native.h 2024-08-08T20:48:49.5883699Z inflating: build/lib/torch/include/ATen/ops/cov_ops.h 2024-08-08T20:48:49.5884564Z inflating: build/lib/torch/include/ATen/ops/cross.h 2024-08-08T20:48:49.5885769Z inflating: build/lib/torch/include/ATen/ops/cross_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5886864Z inflating: build/lib/torch/include/ATen/ops/cross_entropy_loss.h 2024-08-08T20:48:49.5888059Z inflating: build/lib/torch/include/ATen/ops/cross_entropy_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5889611Z inflating: build/lib/torch/include/ATen/ops/cross_entropy_loss_native.h 2024-08-08T20:48:49.5890627Z inflating: build/lib/torch/include/ATen/ops/cross_entropy_loss_ops.h 2024-08-08T20:48:49.5891534Z inflating: build/lib/torch/include/ATen/ops/cross_native.h 2024-08-08T20:48:49.5892311Z inflating: build/lib/torch/include/ATen/ops/cross_ops.h 2024-08-08T20:48:49.5893075Z inflating: build/lib/torch/include/ATen/ops/crow_indices.h 2024-08-08T20:48:49.5894132Z inflating: build/lib/torch/include/ATen/ops/crow_indices_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5895274Z inflating: build/lib/torch/include/ATen/ops/crow_indices_copy.h 2024-08-08T20:48:49.5896499Z inflating: build/lib/torch/include/ATen/ops/crow_indices_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5898074Z inflating: build/lib/torch/include/ATen/ops/crow_indices_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5899432Z inflating: build/lib/torch/include/ATen/ops/crow_indices_copy_native.h 2024-08-08T20:48:49.5900537Z inflating: build/lib/torch/include/ATen/ops/crow_indices_copy_ops.h 2024-08-08T20:48:49.5901423Z inflating: build/lib/torch/include/ATen/ops/crow_indices_native.h 2024-08-08T20:48:49.5902381Z inflating: build/lib/torch/include/ATen/ops/crow_indices_ops.h 2024-08-08T20:48:49.5903187Z inflating: build/lib/torch/include/ATen/ops/ctc_loss.h 2024-08-08T20:48:49.5904205Z inflating: build/lib/torch/include/ATen/ops/ctc_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5905293Z inflating: build/lib/torch/include/ATen/ops/ctc_loss_native.h 2024-08-08T20:48:49.5906152Z inflating: build/lib/torch/include/ATen/ops/ctc_loss_ops.h 2024-08-08T20:48:49.5907083Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator.h 2024-08-08T20:48:49.5908203Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator_backward.h 2024-08-08T20:48:49.5909666Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5911225Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator_backward_cuda_dispatch.h 2024-08-08T20:48:49.5912544Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator_backward_native.h 2024-08-08T20:48:49.5913830Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator_backward_ops.h 2024-08-08T20:48:49.5915269Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5916725Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator_cuda_dispatch.h 2024-08-08T20:48:49.5917931Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator_native.h 2024-08-08T20:48:49.5919046Z inflating: build/lib/torch/include/ATen/ops/cudnn_affine_grid_generator_ops.h 2024-08-08T20:48:49.5920066Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm.h 2024-08-08T20:48:49.5921035Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm_backward.h 2024-08-08T20:48:49.5922330Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5923766Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm_backward_cuda_dispatch.h 2024-08-08T20:48:49.5924952Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm_backward_native.h 2024-08-08T20:48:49.5926043Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm_backward_ops.h 2024-08-08T20:48:49.5927294Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5928577Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm_cuda_dispatch.h 2024-08-08T20:48:49.5929634Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm_native.h 2024-08-08T20:48:49.5930692Z inflating: build/lib/torch/include/ATen/ops/cudnn_batch_norm_ops.h 2024-08-08T20:48:49.5931613Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution.h 2024-08-08T20:48:49.5932624Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_add_relu.h 2024-08-08T20:48:49.5933936Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_add_relu_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5935352Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_add_relu_cuda_dispatch.h 2024-08-08T20:48:49.5936559Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_add_relu_native.h 2024-08-08T20:48:49.5937700Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_add_relu_ops.h 2024-08-08T20:48:49.5938802Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_cuda_dispatch.h 2024-08-08T20:48:49.5939878Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_native.h 2024-08-08T20:48:49.5940883Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_ops.h 2024-08-08T20:48:49.5941848Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_relu.h 2024-08-08T20:48:49.5943192Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_relu_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5944571Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_relu_cuda_dispatch.h 2024-08-08T20:48:49.5945703Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_relu_native.h 2024-08-08T20:48:49.5946718Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_relu_ops.h 2024-08-08T20:48:49.5947827Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_transpose.h 2024-08-08T20:48:49.5949145Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_transpose_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5950563Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_transpose_cuda_dispatch.h 2024-08-08T20:48:49.5951748Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_transpose_native.h 2024-08-08T20:48:49.5952923Z inflating: build/lib/torch/include/ATen/ops/cudnn_convolution_transpose_ops.h 2024-08-08T20:48:49.5953946Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler.h 2024-08-08T20:48:49.5954923Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler_backward.h 2024-08-08T20:48:49.5956277Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5957718Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler_backward_cuda_dispatch.h 2024-08-08T20:48:49.5959061Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler_backward_native.h 2024-08-08T20:48:49.5960236Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler_backward_ops.h 2024-08-08T20:48:49.5961566Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5962869Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler_cuda_dispatch.h 2024-08-08T20:48:49.5963952Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler_native.h 2024-08-08T20:48:49.5964965Z inflating: build/lib/torch/include/ATen/ops/cudnn_grid_sampler_ops.h 2024-08-08T20:48:49.5965923Z inflating: build/lib/torch/include/ATen/ops/cudnn_is_acceptable.h 2024-08-08T20:48:49.5967119Z inflating: build/lib/torch/include/ATen/ops/cudnn_is_acceptable_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5968375Z inflating: build/lib/torch/include/ATen/ops/cudnn_is_acceptable_native.h 2024-08-08T20:48:49.5969355Z inflating: build/lib/torch/include/ATen/ops/cudnn_is_acceptable_ops.h 2024-08-08T20:48:49.5970243Z inflating: build/lib/torch/include/ATen/ops/cummax.h 2024-08-08T20:48:49.5971261Z inflating: build/lib/torch/include/ATen/ops/cummax_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5972701Z inflating: build/lib/torch/include/ATen/ops/cummax_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5973794Z inflating: build/lib/torch/include/ATen/ops/cummax_native.h 2024-08-08T20:48:49.5974608Z inflating: build/lib/torch/include/ATen/ops/cummax_ops.h 2024-08-08T20:48:49.5975439Z inflating: build/lib/torch/include/ATen/ops/cummaxmin_backward.h 2024-08-08T20:48:49.5976620Z inflating: build/lib/torch/include/ATen/ops/cummaxmin_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5977891Z inflating: build/lib/torch/include/ATen/ops/cummaxmin_backward_native.h 2024-08-08T20:48:49.5978892Z inflating: build/lib/torch/include/ATen/ops/cummaxmin_backward_ops.h 2024-08-08T20:48:49.5979775Z inflating: build/lib/torch/include/ATen/ops/cummin.h 2024-08-08T20:48:49.5980696Z inflating: build/lib/torch/include/ATen/ops/cummin_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.5982079Z inflating: build/lib/torch/include/ATen/ops/cummin_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5983162Z inflating: build/lib/torch/include/ATen/ops/cummin_native.h 2024-08-08T20:48:49.5983915Z inflating: build/lib/torch/include/ATen/ops/cumprod.h 2024-08-08T20:48:49.5984837Z inflating: build/lib/torch/include/ATen/ops/cumprod_backward.h 2024-08-08T20:48:49.5986046Z inflating: build/lib/torch/include/ATen/ops/cumprod_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5987287Z inflating: build/lib/torch/include/ATen/ops/cumprod_backward_native.h 2024-08-08T20:48:49.5988253Z inflating: build/lib/torch/include/ATen/ops/cumprod_backward_ops.h 2024-08-08T20:48:49.5989556Z inflating: build/lib/torch/include/ATen/ops/cumprod_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.5990982Z inflating: build/lib/torch/include/ATen/ops/cumprod_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.5992057Z inflating: build/lib/torch/include/ATen/ops/cumprod_cpu_dispatch.h 2024-08-08T20:48:49.5993026Z inflating: build/lib/torch/include/ATen/ops/cumprod_cuda_dispatch.h 2024-08-08T20:48:49.5993962Z inflating: build/lib/torch/include/ATen/ops/cumprod_meta.h 2024-08-08T20:48:49.5994860Z inflating: build/lib/torch/include/ATen/ops/cumprod_meta_dispatch.h 2024-08-08T20:48:49.5995754Z inflating: build/lib/torch/include/ATen/ops/cumprod_native.h 2024-08-08T20:48:49.5996595Z inflating: build/lib/torch/include/ATen/ops/cumprod_ops.h 2024-08-08T20:48:49.5997605Z inflating: build/lib/torch/include/ATen/ops/cumsum.h 2024-08-08T20:48:49.5998869Z inflating: build/lib/torch/include/ATen/ops/cumsum_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6000230Z inflating: build/lib/torch/include/ATen/ops/cumsum_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6001321Z inflating: build/lib/torch/include/ATen/ops/cumsum_cpu_dispatch.h 2024-08-08T20:48:49.6002330Z inflating: build/lib/torch/include/ATen/ops/cumsum_cuda_dispatch.h 2024-08-08T20:48:49.6003186Z inflating: build/lib/torch/include/ATen/ops/cumsum_meta.h 2024-08-08T20:48:49.6003551Z inflating: build/lib/torch/include/ATen/ops/cumsum_meta_dispatch.h 2024-08-08T20:48:49.6003918Z inflating: build/lib/torch/include/ATen/ops/cumsum_native.h 2024-08-08T20:48:49.6004214Z inflating: build/lib/torch/include/ATen/ops/cumsum_ops.h 2024-08-08T20:48:49.6004600Z inflating: build/lib/torch/include/ATen/ops/cumulative_trapezoid.h 2024-08-08T20:48:49.6005266Z inflating: build/lib/torch/include/ATen/ops/cumulative_trapezoid_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6005701Z inflating: build/lib/torch/include/ATen/ops/cumulative_trapezoid_native.h 2024-08-08T20:48:49.6006141Z inflating: build/lib/torch/include/ATen/ops/cumulative_trapezoid_ops.h 2024-08-08T20:48:49.6006389Z inflating: build/lib/torch/include/ATen/ops/data.h 2024-08-08T20:48:49.6006914Z inflating: build/lib/torch/include/ATen/ops/data_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6007327Z inflating: build/lib/torch/include/ATen/ops/data_native.h 2024-08-08T20:48:49.6007595Z inflating: build/lib/torch/include/ATen/ops/data_ops.h 2024-08-08T20:48:49.6007914Z inflating: build/lib/torch/include/ATen/ops/deg2rad.h 2024-08-08T20:48:49.6009038Z inflating: build/lib/torch/include/ATen/ops/deg2rad_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6009740Z inflating: build/lib/torch/include/ATen/ops/deg2rad_native.h 2024-08-08T20:48:49.6010747Z inflating: build/lib/torch/include/ATen/ops/deg2rad_ops.h 2024-08-08T20:48:49.6011626Z inflating: build/lib/torch/include/ATen/ops/dense_dim.h 2024-08-08T20:48:49.6012896Z inflating: build/lib/torch/include/ATen/ops/dense_dim_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6013600Z inflating: build/lib/torch/include/ATen/ops/dense_dim_native.h 2024-08-08T20:48:49.6014622Z inflating: build/lib/torch/include/ATen/ops/dense_dim_ops.h 2024-08-08T20:48:49.6015570Z inflating: build/lib/torch/include/ATen/ops/dequantize.h 2024-08-08T20:48:49.6017477Z inflating: build/lib/torch/include/ATen/ops/dequantize_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6018497Z inflating: build/lib/torch/include/ATen/ops/dequantize_cpu_dispatch.h 2024-08-08T20:48:49.6019372Z inflating: build/lib/torch/include/ATen/ops/dequantize_cuda_dispatch.h 2024-08-08T20:48:49.6020229Z inflating: build/lib/torch/include/ATen/ops/dequantize_native.h 2024-08-08T20:48:49.6021352Z inflating: build/lib/torch/include/ATen/ops/dequantize_ops.h 2024-08-08T20:48:49.6022206Z inflating: build/lib/torch/include/ATen/ops/det.h 2024-08-08T20:48:49.6023497Z inflating: build/lib/torch/include/ATen/ops/det_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6024220Z inflating: build/lib/torch/include/ATen/ops/det_native.h 2024-08-08T20:48:49.6025182Z inflating: build/lib/torch/include/ATen/ops/det_ops.h 2024-08-08T20:48:49.6026123Z inflating: build/lib/torch/include/ATen/ops/detach.h 2024-08-08T20:48:49.6027431Z inflating: build/lib/torch/include/ATen/ops/detach_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6028214Z inflating: build/lib/torch/include/ATen/ops/detach_copy.h 2024-08-08T20:48:49.6029461Z inflating: build/lib/torch/include/ATen/ops/detach_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6030572Z inflating: build/lib/torch/include/ATen/ops/detach_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6031143Z inflating: build/lib/torch/include/ATen/ops/detach_copy_native.h 2024-08-08T20:48:49.6032160Z inflating: build/lib/torch/include/ATen/ops/detach_copy_ops.h 2024-08-08T20:48:49.6033043Z inflating: build/lib/torch/include/ATen/ops/detach_native.h 2024-08-08T20:48:49.6034057Z inflating: build/lib/torch/include/ATen/ops/detach_ops.h 2024-08-08T20:48:49.6034970Z inflating: build/lib/torch/include/ATen/ops/diag.h 2024-08-08T20:48:49.6036243Z inflating: build/lib/torch/include/ATen/ops/diag_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6037045Z inflating: build/lib/torch/include/ATen/ops/diag_embed.h 2024-08-08T20:48:49.6038309Z inflating: build/lib/torch/include/ATen/ops/diag_embed_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6039415Z inflating: build/lib/torch/include/ATen/ops/diag_embed_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6040040Z inflating: build/lib/torch/include/ATen/ops/diag_embed_native.h 2024-08-08T20:48:49.6041044Z inflating: build/lib/torch/include/ATen/ops/diag_embed_ops.h 2024-08-08T20:48:49.6042016Z inflating: build/lib/torch/include/ATen/ops/diag_native.h 2024-08-08T20:48:49.6043031Z inflating: build/lib/torch/include/ATen/ops/diag_ops.h 2024-08-08T20:48:49.6043968Z inflating: build/lib/torch/include/ATen/ops/diagflat.h 2024-08-08T20:48:49.6045261Z inflating: build/lib/torch/include/ATen/ops/diagflat_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6045932Z inflating: build/lib/torch/include/ATen/ops/diagflat_native.h 2024-08-08T20:48:49.6046956Z inflating: build/lib/torch/include/ATen/ops/diagflat_ops.h 2024-08-08T20:48:49.6047842Z inflating: build/lib/torch/include/ATen/ops/diagonal.h 2024-08-08T20:48:49.6049597Z inflating: build/lib/torch/include/ATen/ops/diagonal_backward.h 2024-08-08T20:48:49.6050722Z inflating: build/lib/torch/include/ATen/ops/diagonal_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6051433Z inflating: build/lib/torch/include/ATen/ops/diagonal_backward_native.h 2024-08-08T20:48:49.6052435Z inflating: build/lib/torch/include/ATen/ops/diagonal_backward_ops.h 2024-08-08T20:48:49.6053666Z inflating: build/lib/torch/include/ATen/ops/diagonal_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6054672Z inflating: build/lib/torch/include/ATen/ops/diagonal_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6055445Z inflating: build/lib/torch/include/ATen/ops/diagonal_copy.h 2024-08-08T20:48:49.6056716Z inflating: build/lib/torch/include/ATen/ops/diagonal_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6057865Z inflating: build/lib/torch/include/ATen/ops/diagonal_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6058902Z inflating: build/lib/torch/include/ATen/ops/diagonal_copy_native.h 2024-08-08T20:48:49.6060676Z inflating: build/lib/torch/include/ATen/ops/diagonal_copy_ops.h 2024-08-08T20:48:49.6061549Z inflating: build/lib/torch/include/ATen/ops/diagonal_native.h 2024-08-08T20:48:49.6062523Z inflating: build/lib/torch/include/ATen/ops/diagonal_ops.h 2024-08-08T20:48:49.6063524Z inflating: build/lib/torch/include/ATen/ops/diagonal_scatter.h 2024-08-08T20:48:49.6064820Z inflating: build/lib/torch/include/ATen/ops/diagonal_scatter_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6066002Z inflating: build/lib/torch/include/ATen/ops/diagonal_scatter_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6066521Z inflating: build/lib/torch/include/ATen/ops/diagonal_scatter_native.h 2024-08-08T20:48:49.6067603Z inflating: build/lib/torch/include/ATen/ops/diagonal_scatter_ops.h 2024-08-08T20:48:49.6068535Z inflating: build/lib/torch/include/ATen/ops/diff.h 2024-08-08T20:48:49.6069810Z inflating: build/lib/torch/include/ATen/ops/diff_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6070552Z inflating: build/lib/torch/include/ATen/ops/diff_native.h 2024-08-08T20:48:49.6071897Z inflating: build/lib/torch/include/ATen/ops/diff_ops.h 2024-08-08T20:48:49.6072629Z inflating: build/lib/torch/include/ATen/ops/digamma.h 2024-08-08T20:48:49.6074034Z inflating: build/lib/torch/include/ATen/ops/digamma_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6074742Z inflating: build/lib/torch/include/ATen/ops/digamma_cpu_dispatch.h 2024-08-08T20:48:49.6075649Z inflating: build/lib/torch/include/ATen/ops/digamma_cuda_dispatch.h 2024-08-08T20:48:49.6076537Z inflating: build/lib/torch/include/ATen/ops/digamma_meta.h 2024-08-08T20:48:49.6077573Z inflating: build/lib/torch/include/ATen/ops/digamma_meta_dispatch.h 2024-08-08T20:48:49.6078487Z inflating: build/lib/torch/include/ATen/ops/digamma_native.h 2024-08-08T20:48:49.6079523Z inflating: build/lib/torch/include/ATen/ops/digamma_ops.h 2024-08-08T20:48:49.6080420Z inflating: build/lib/torch/include/ATen/ops/dist.h 2024-08-08T20:48:49.6081760Z inflating: build/lib/torch/include/ATen/ops/dist_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6082635Z inflating: build/lib/torch/include/ATen/ops/dist_native.h 2024-08-08T20:48:49.6084731Z inflating: build/lib/torch/include/ATen/ops/dist_ops.h 2024-08-08T20:48:49.6085604Z inflating: build/lib/torch/include/ATen/ops/div.h 2024-08-08T20:48:49.6086966Z inflating: build/lib/torch/include/ATen/ops/div_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6088095Z inflating: build/lib/torch/include/ATen/ops/div_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6088889Z inflating: build/lib/torch/include/ATen/ops/div_cpu_dispatch.h 2024-08-08T20:48:49.6089816Z inflating: build/lib/torch/include/ATen/ops/div_cuda_dispatch.h 2024-08-08T20:48:49.6090728Z inflating: build/lib/torch/include/ATen/ops/div_meta.h 2024-08-08T20:48:49.6091766Z inflating: build/lib/torch/include/ATen/ops/div_meta_dispatch.h 2024-08-08T20:48:49.6092791Z inflating: build/lib/torch/include/ATen/ops/div_native.h 2024-08-08T20:48:49.6094441Z inflating: build/lib/torch/include/ATen/ops/div_ops.h 2024-08-08T20:48:49.6095256Z inflating: build/lib/torch/include/ATen/ops/divide.h 2024-08-08T20:48:49.6096868Z inflating: build/lib/torch/include/ATen/ops/divide_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6097324Z inflating: build/lib/torch/include/ATen/ops/divide_native.h 2024-08-08T20:48:49.6099001Z inflating: build/lib/torch/include/ATen/ops/divide_ops.h 2024-08-08T20:48:49.6099597Z inflating: build/lib/torch/include/ATen/ops/dot.h 2024-08-08T20:48:49.6101232Z inflating: build/lib/torch/include/ATen/ops/dot_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6101756Z inflating: build/lib/torch/include/ATen/ops/dot_cpu_dispatch.h 2024-08-08T20:48:49.6102898Z inflating: build/lib/torch/include/ATen/ops/dot_cuda_dispatch.h 2024-08-08T20:48:49.6103696Z inflating: build/lib/torch/include/ATen/ops/dot_native.h 2024-08-08T20:48:49.6104715Z inflating: build/lib/torch/include/ATen/ops/dot_ops.h 2024-08-08T20:48:49.6105603Z inflating: build/lib/torch/include/ATen/ops/dropout.h 2024-08-08T20:48:49.6106912Z inflating: build/lib/torch/include/ATen/ops/dropout_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6107618Z inflating: build/lib/torch/include/ATen/ops/dropout_native.h 2024-08-08T20:48:49.6108623Z inflating: build/lib/torch/include/ATen/ops/dropout_ops.h 2024-08-08T20:48:49.6109543Z inflating: build/lib/torch/include/ATen/ops/dsplit.h 2024-08-08T20:48:49.6110814Z inflating: build/lib/torch/include/ATen/ops/dsplit_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6111575Z inflating: build/lib/torch/include/ATen/ops/dsplit_native.h 2024-08-08T20:48:49.6112599Z inflating: build/lib/torch/include/ATen/ops/dsplit_ops.h 2024-08-08T20:48:49.6113505Z inflating: build/lib/torch/include/ATen/ops/dstack.h 2024-08-08T20:48:49.6114890Z inflating: build/lib/torch/include/ATen/ops/dstack_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6115611Z inflating: build/lib/torch/include/ATen/ops/dstack_native.h 2024-08-08T20:48:49.6117008Z inflating: build/lib/torch/include/ATen/ops/dstack_ops.h 2024-08-08T20:48:49.6117721Z inflating: build/lib/torch/include/ATen/ops/einsum.h 2024-08-08T20:48:49.6119010Z inflating: build/lib/torch/include/ATen/ops/einsum_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6119702Z inflating: build/lib/torch/include/ATen/ops/einsum_native.h 2024-08-08T20:48:49.6120655Z inflating: build/lib/torch/include/ATen/ops/einsum_ops.h 2024-08-08T20:48:49.6121615Z inflating: build/lib/torch/include/ATen/ops/elu.h 2024-08-08T20:48:49.6122710Z inflating: build/lib/torch/include/ATen/ops/elu_backward.h 2024-08-08T20:48:49.6124130Z inflating: build/lib/torch/include/ATen/ops/elu_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6124900Z inflating: build/lib/torch/include/ATen/ops/elu_backward_cpu_dispatch.h 2024-08-08T20:48:49.6125934Z inflating: build/lib/torch/include/ATen/ops/elu_backward_cuda_dispatch.h 2024-08-08T20:48:49.6126856Z inflating: build/lib/torch/include/ATen/ops/elu_backward_meta.h 2024-08-08T20:48:49.6127958Z inflating: build/lib/torch/include/ATen/ops/elu_backward_meta_dispatch.h 2024-08-08T20:48:49.6128816Z inflating: build/lib/torch/include/ATen/ops/elu_backward_native.h 2024-08-08T20:48:49.6129893Z inflating: build/lib/torch/include/ATen/ops/elu_backward_ops.h 2024-08-08T20:48:49.6131199Z inflating: build/lib/torch/include/ATen/ops/elu_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6131997Z inflating: build/lib/torch/include/ATen/ops/elu_cpu_dispatch.h 2024-08-08T20:48:49.6132911Z inflating: build/lib/torch/include/ATen/ops/elu_cuda_dispatch.h 2024-08-08T20:48:49.6133760Z inflating: build/lib/torch/include/ATen/ops/elu_meta.h 2024-08-08T20:48:49.6134841Z inflating: build/lib/torch/include/ATen/ops/elu_meta_dispatch.h 2024-08-08T20:48:49.6135734Z inflating: build/lib/torch/include/ATen/ops/elu_native.h 2024-08-08T20:48:49.6136820Z inflating: build/lib/torch/include/ATen/ops/elu_ops.h 2024-08-08T20:48:49.6138325Z inflating: build/lib/torch/include/ATen/ops/embedding.h 2024-08-08T20:48:49.6139098Z inflating: build/lib/torch/include/ATen/ops/embedding_backward.h 2024-08-08T20:48:49.6140829Z inflating: build/lib/torch/include/ATen/ops/embedding_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6141394Z inflating: build/lib/torch/include/ATen/ops/embedding_backward_native.h 2024-08-08T20:48:49.6142381Z inflating: build/lib/torch/include/ATen/ops/embedding_backward_ops.h 2024-08-08T20:48:49.6143292Z inflating: build/lib/torch/include/ATen/ops/embedding_bag.h 2024-08-08T20:48:49.6144753Z inflating: build/lib/torch/include/ATen/ops/embedding_bag_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6145523Z inflating: build/lib/torch/include/ATen/ops/embedding_bag_native.h 2024-08-08T20:48:49.6146559Z inflating: build/lib/torch/include/ATen/ops/embedding_bag_ops.h 2024-08-08T20:48:49.6147822Z inflating: build/lib/torch/include/ATen/ops/embedding_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6148912Z inflating: build/lib/torch/include/ATen/ops/embedding_dense_backward.h 2024-08-08T20:48:49.6150393Z inflating: build/lib/torch/include/ATen/ops/embedding_dense_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6152001Z inflating: build/lib/torch/include/ATen/ops/embedding_dense_backward_cpu_dispatch.h 2024-08-08T20:48:49.6152847Z inflating: build/lib/torch/include/ATen/ops/embedding_dense_backward_cuda_dispatch.h 2024-08-08T20:48:49.6153764Z inflating: build/lib/torch/include/ATen/ops/embedding_dense_backward_native.h 2024-08-08T20:48:49.6154902Z inflating: build/lib/torch/include/ATen/ops/embedding_dense_backward_ops.h 2024-08-08T20:48:49.6155748Z inflating: build/lib/torch/include/ATen/ops/embedding_native.h 2024-08-08T20:48:49.6156815Z inflating: build/lib/torch/include/ATen/ops/embedding_ops.h 2024-08-08T20:48:49.6157837Z inflating: build/lib/torch/include/ATen/ops/embedding_renorm.h 2024-08-08T20:48:49.6159346Z inflating: build/lib/torch/include/ATen/ops/embedding_renorm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6160428Z inflating: build/lib/torch/include/ATen/ops/embedding_renorm_cpu_dispatch.h 2024-08-08T20:48:49.6161378Z inflating: build/lib/torch/include/ATen/ops/embedding_renorm_cuda_dispatch.h 2024-08-08T20:48:49.6162366Z inflating: build/lib/torch/include/ATen/ops/embedding_renorm_meta_dispatch.h 2024-08-08T20:48:49.6163246Z inflating: build/lib/torch/include/ATen/ops/embedding_renorm_native.h 2024-08-08T20:48:49.6164293Z inflating: build/lib/torch/include/ATen/ops/embedding_renorm_ops.h 2024-08-08T20:48:49.6165295Z inflating: build/lib/torch/include/ATen/ops/embedding_sparse_backward.h 2024-08-08T20:48:49.6166545Z inflating: build/lib/torch/include/ATen/ops/embedding_sparse_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6167290Z inflating: build/lib/torch/include/ATen/ops/embedding_sparse_backward_native.h 2024-08-08T20:48:49.6168385Z inflating: build/lib/torch/include/ATen/ops/embedding_sparse_backward_ops.h 2024-08-08T20:48:49.6169884Z inflating: build/lib/torch/include/ATen/ops/empty.h 2024-08-08T20:48:49.6170870Z inflating: build/lib/torch/include/ATen/ops/empty_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6171956Z inflating: build/lib/torch/include/ATen/ops/empty_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6172886Z inflating: build/lib/torch/include/ATen/ops/empty_cpu_dispatch.h 2024-08-08T20:48:49.6173829Z inflating: build/lib/torch/include/ATen/ops/empty_cuda_dispatch.h 2024-08-08T20:48:49.6174796Z inflating: build/lib/torch/include/ATen/ops/empty_like.h 2024-08-08T20:48:49.6176115Z inflating: build/lib/torch/include/ATen/ops/empty_like_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6176892Z inflating: build/lib/torch/include/ATen/ops/empty_like_native.h 2024-08-08T20:48:49.6177960Z inflating: build/lib/torch/include/ATen/ops/empty_like_ops.h 2024-08-08T20:48:49.6178998Z inflating: build/lib/torch/include/ATen/ops/empty_meta_dispatch.h 2024-08-08T20:48:49.6180068Z inflating: build/lib/torch/include/ATen/ops/empty_native.h 2024-08-08T20:48:49.6181549Z inflating: build/lib/torch/include/ATen/ops/empty_ops.h 2024-08-08T20:48:49.6182655Z inflating: build/lib/torch/include/ATen/ops/empty_permuted.h 2024-08-08T20:48:49.6184032Z inflating: build/lib/torch/include/ATen/ops/empty_permuted_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6184912Z inflating: build/lib/torch/include/ATen/ops/empty_permuted_native.h 2024-08-08T20:48:49.6186122Z inflating: build/lib/torch/include/ATen/ops/empty_permuted_ops.h 2024-08-08T20:48:49.6187042Z inflating: build/lib/torch/include/ATen/ops/empty_quantized.h 2024-08-08T20:48:49.6188333Z inflating: build/lib/torch/include/ATen/ops/empty_quantized_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6189072Z inflating: build/lib/torch/include/ATen/ops/empty_quantized_native.h 2024-08-08T20:48:49.6190164Z inflating: build/lib/torch/include/ATen/ops/empty_quantized_ops.h 2024-08-08T20:48:49.6191697Z inflating: build/lib/torch/include/ATen/ops/empty_strided.h 2024-08-08T20:48:49.6192796Z inflating: build/lib/torch/include/ATen/ops/empty_strided_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6193721Z inflating: build/lib/torch/include/ATen/ops/empty_strided_cpu_dispatch.h 2024-08-08T20:48:49.6194723Z inflating: build/lib/torch/include/ATen/ops/empty_strided_cuda_dispatch.h 2024-08-08T20:48:49.6195721Z inflating: build/lib/torch/include/ATen/ops/empty_strided_meta_dispatch.h 2024-08-08T20:48:49.6196637Z inflating: build/lib/torch/include/ATen/ops/empty_strided_native.h 2024-08-08T20:48:49.6197722Z inflating: build/lib/torch/include/ATen/ops/empty_strided_ops.h 2024-08-08T20:48:49.6198659Z inflating: build/lib/torch/include/ATen/ops/eq.h 2024-08-08T20:48:49.6200031Z inflating: build/lib/torch/include/ATen/ops/eq_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6200746Z inflating: build/lib/torch/include/ATen/ops/eq_cpu_dispatch.h 2024-08-08T20:48:49.6202295Z inflating: build/lib/torch/include/ATen/ops/eq_cuda_dispatch.h 2024-08-08T20:48:49.6203014Z inflating: build/lib/torch/include/ATen/ops/eq_meta.h 2024-08-08T20:48:49.6204079Z inflating: build/lib/torch/include/ATen/ops/eq_meta_dispatch.h 2024-08-08T20:48:49.6205047Z inflating: build/lib/torch/include/ATen/ops/eq_native.h 2024-08-08T20:48:49.6206425Z inflating: build/lib/torch/include/ATen/ops/eq_ops.h 2024-08-08T20:48:49.6207096Z inflating: build/lib/torch/include/ATen/ops/equal.h 2024-08-08T20:48:49.6208223Z inflating: build/lib/torch/include/ATen/ops/equal_cpu_dispatch.h 2024-08-08T20:48:49.6209193Z inflating: build/lib/torch/include/ATen/ops/equal_cuda_dispatch.h 2024-08-08T20:48:49.6210065Z inflating: build/lib/torch/include/ATen/ops/equal_native.h 2024-08-08T20:48:49.6211106Z inflating: build/lib/torch/include/ATen/ops/equal_ops.h 2024-08-08T20:48:49.6212048Z inflating: build/lib/torch/include/ATen/ops/erf.h 2024-08-08T20:48:49.6213441Z inflating: build/lib/torch/include/ATen/ops/erf_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6214141Z inflating: build/lib/torch/include/ATen/ops/erf_cpu_dispatch.h 2024-08-08T20:48:49.6215096Z inflating: build/lib/torch/include/ATen/ops/erf_cuda_dispatch.h 2024-08-08T20:48:49.6216022Z inflating: build/lib/torch/include/ATen/ops/erf_meta.h 2024-08-08T20:48:49.6217026Z inflating: build/lib/torch/include/ATen/ops/erf_meta_dispatch.h 2024-08-08T20:48:49.6218157Z inflating: build/lib/torch/include/ATen/ops/erf_native.h 2024-08-08T20:48:49.6219949Z inflating: build/lib/torch/include/ATen/ops/erf_ops.h 2024-08-08T20:48:49.6220530Z inflating: build/lib/torch/include/ATen/ops/erfc.h 2024-08-08T20:48:49.6222325Z inflating: build/lib/torch/include/ATen/ops/erfc_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6222898Z inflating: build/lib/torch/include/ATen/ops/erfc_cpu_dispatch.h 2024-08-08T20:48:49.6223801Z inflating: build/lib/torch/include/ATen/ops/erfc_cuda_dispatch.h 2024-08-08T20:48:49.6224667Z inflating: build/lib/torch/include/ATen/ops/erfc_meta.h 2024-08-08T20:48:49.6225724Z inflating: build/lib/torch/include/ATen/ops/erfc_meta_dispatch.h 2024-08-08T20:48:49.6226631Z inflating: build/lib/torch/include/ATen/ops/erfc_native.h 2024-08-08T20:48:49.6227634Z inflating: build/lib/torch/include/ATen/ops/erfc_ops.h 2024-08-08T20:48:49.6228594Z inflating: build/lib/torch/include/ATen/ops/erfinv.h 2024-08-08T20:48:49.6230128Z inflating: build/lib/torch/include/ATen/ops/erfinv_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6230767Z inflating: build/lib/torch/include/ATen/ops/erfinv_cpu_dispatch.h 2024-08-08T20:48:49.6231680Z inflating: build/lib/torch/include/ATen/ops/erfinv_cuda_dispatch.h 2024-08-08T20:48:49.6232569Z inflating: build/lib/torch/include/ATen/ops/erfinv_meta.h 2024-08-08T20:48:49.6233589Z inflating: build/lib/torch/include/ATen/ops/erfinv_meta_dispatch.h 2024-08-08T20:48:49.6234525Z inflating: build/lib/torch/include/ATen/ops/erfinv_native.h 2024-08-08T20:48:49.6235568Z inflating: build/lib/torch/include/ATen/ops/erfinv_ops.h 2024-08-08T20:48:49.6236465Z inflating: build/lib/torch/include/ATen/ops/exp.h 2024-08-08T20:48:49.6237436Z inflating: build/lib/torch/include/ATen/ops/exp2.h 2024-08-08T20:48:49.6238878Z inflating: build/lib/torch/include/ATen/ops/exp2_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6239625Z inflating: build/lib/torch/include/ATen/ops/exp2_cpu_dispatch.h 2024-08-08T20:48:49.6240580Z inflating: build/lib/torch/include/ATen/ops/exp2_cuda_dispatch.h 2024-08-08T20:48:49.6241491Z inflating: build/lib/torch/include/ATen/ops/exp2_meta.h 2024-08-08T20:48:49.6242467Z inflating: build/lib/torch/include/ATen/ops/exp2_meta_dispatch.h 2024-08-08T20:48:49.6243356Z inflating: build/lib/torch/include/ATen/ops/exp2_native.h 2024-08-08T20:48:49.6244401Z inflating: build/lib/torch/include/ATen/ops/exp2_ops.h 2024-08-08T20:48:49.6245743Z inflating: build/lib/torch/include/ATen/ops/exp_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6246448Z inflating: build/lib/torch/include/ATen/ops/exp_cpu_dispatch.h 2024-08-08T20:48:49.6247509Z inflating: build/lib/torch/include/ATen/ops/exp_cuda_dispatch.h 2024-08-08T20:48:49.6248357Z inflating: build/lib/torch/include/ATen/ops/exp_meta.h 2024-08-08T20:48:49.6249445Z inflating: build/lib/torch/include/ATen/ops/exp_meta_dispatch.h 2024-08-08T20:48:49.6250487Z inflating: build/lib/torch/include/ATen/ops/exp_native.h 2024-08-08T20:48:49.6251868Z inflating: build/lib/torch/include/ATen/ops/exp_ops.h 2024-08-08T20:48:49.6252555Z inflating: build/lib/torch/include/ATen/ops/expand.h 2024-08-08T20:48:49.6253531Z inflating: build/lib/torch/include/ATen/ops/expand_as.h 2024-08-08T20:48:49.6254879Z inflating: build/lib/torch/include/ATen/ops/expand_as_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6255611Z inflating: build/lib/torch/include/ATen/ops/expand_as_native.h 2024-08-08T20:48:49.6256612Z inflating: build/lib/torch/include/ATen/ops/expand_as_ops.h 2024-08-08T20:48:49.6257834Z inflating: build/lib/torch/include/ATen/ops/expand_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6258996Z inflating: build/lib/torch/include/ATen/ops/expand_copy.h 2024-08-08T20:48:49.6260304Z inflating: build/lib/torch/include/ATen/ops/expand_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6261448Z inflating: build/lib/torch/include/ATen/ops/expand_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6262041Z inflating: build/lib/torch/include/ATen/ops/expand_copy_native.h 2024-08-08T20:48:49.6263079Z inflating: build/lib/torch/include/ATen/ops/expand_copy_ops.h 2024-08-08T20:48:49.6264042Z inflating: build/lib/torch/include/ATen/ops/expand_native.h 2024-08-08T20:48:49.6264998Z inflating: build/lib/torch/include/ATen/ops/expand_ops.h 2024-08-08T20:48:49.6265965Z inflating: build/lib/torch/include/ATen/ops/expm1.h 2024-08-08T20:48:49.6267375Z inflating: build/lib/torch/include/ATen/ops/expm1_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6268081Z inflating: build/lib/torch/include/ATen/ops/expm1_cpu_dispatch.h 2024-08-08T20:48:49.6269082Z inflating: build/lib/torch/include/ATen/ops/expm1_cuda_dispatch.h 2024-08-08T20:48:49.6269999Z inflating: build/lib/torch/include/ATen/ops/expm1_meta.h 2024-08-08T20:48:49.6271127Z inflating: build/lib/torch/include/ATen/ops/expm1_meta_dispatch.h 2024-08-08T20:48:49.6271970Z inflating: build/lib/torch/include/ATen/ops/expm1_native.h 2024-08-08T20:48:49.6273023Z inflating: build/lib/torch/include/ATen/ops/expm1_ops.h 2024-08-08T20:48:49.6273986Z inflating: build/lib/torch/include/ATen/ops/exponential.h 2024-08-08T20:48:49.6275322Z inflating: build/lib/torch/include/ATen/ops/exponential_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6276131Z inflating: build/lib/torch/include/ATen/ops/exponential_cpu_dispatch.h 2024-08-08T20:48:49.6277110Z inflating: build/lib/torch/include/ATen/ops/exponential_cuda_dispatch.h 2024-08-08T20:48:49.6278106Z inflating: build/lib/torch/include/ATen/ops/exponential_meta_dispatch.h 2024-08-08T20:48:49.6278942Z inflating: build/lib/torch/include/ATen/ops/exponential_native.h 2024-08-08T20:48:49.6280009Z inflating: build/lib/torch/include/ATen/ops/exponential_ops.h 2024-08-08T20:48:49.6281612Z inflating: build/lib/torch/include/ATen/ops/eye.h 2024-08-08T20:48:49.6282714Z inflating: build/lib/torch/include/ATen/ops/eye_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6283767Z inflating: build/lib/torch/include/ATen/ops/eye_cpu_dispatch.h 2024-08-08T20:48:49.6285504Z inflating: build/lib/torch/include/ATen/ops/eye_cuda_dispatch.h 2024-08-08T20:48:49.6286199Z inflating: build/lib/torch/include/ATen/ops/eye_meta_dispatch.h 2024-08-08T20:48:49.6287204Z inflating: build/lib/torch/include/ATen/ops/eye_native.h 2024-08-08T20:48:49.6288618Z inflating: build/lib/torch/include/ATen/ops/eye_ops.h 2024-08-08T20:48:49.6289621Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine.h 2024-08-08T20:48:49.6290735Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask.h 2024-08-08T20:48:49.6291778Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask_backward.h 2024-08-08T20:48:49.6293118Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6293913Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask_backward_native.h 2024-08-08T20:48:49.6294887Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask_backward_ops.h 2024-08-08T20:48:49.6296068Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6296912Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask_cpu_dispatch.h 2024-08-08T20:48:49.6298016Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask_cuda_dispatch.h 2024-08-08T20:48:49.6298978Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask_native.h 2024-08-08T20:48:49.6300069Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_cachemask_ops.h 2024-08-08T20:48:49.6301235Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6301993Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_native.h 2024-08-08T20:48:49.6303023Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_channel_affine_ops.h 2024-08-08T20:48:49.6303983Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine.h 2024-08-08T20:48:49.6305097Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask.h 2024-08-08T20:48:49.6306123Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask_backward.h 2024-08-08T20:48:49.6307429Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6308320Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask_backward_native.h 2024-08-08T20:48:49.6309199Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask_backward_ops.h 2024-08-08T20:48:49.6310361Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6311205Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask_cpu_dispatch.h 2024-08-08T20:48:49.6312227Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask_cuda_dispatch.h 2024-08-08T20:48:49.6313171Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask_native.h 2024-08-08T20:48:49.6314273Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_cachemask_ops.h 2024-08-08T20:48:49.6315484Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6316239Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_native.h 2024-08-08T20:48:49.6317504Z inflating: build/lib/torch/include/ATen/ops/fake_quantize_per_tensor_affine_ops.h 2024-08-08T20:48:49.6318360Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_fp16_weight.h 2024-08-08T20:48:49.6319698Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_fp16_weight_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6320533Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_fp16_weight_fp32_activation.h 2024-08-08T20:48:49.6321770Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_fp16_weight_fp32_activation_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6322561Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_fp16_weight_fp32_activation_native.h 2024-08-08T20:48:49.6323561Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_fp16_weight_fp32_activation_ops.h 2024-08-08T20:48:49.6324483Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_fp16_weight_native.h 2024-08-08T20:48:49.6325454Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_fp16_weight_ops.h 2024-08-08T20:48:49.6326425Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_int8_weight.h 2024-08-08T20:48:49.6327746Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_int8_weight_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6328585Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_int8_weight_fp32_activation.h 2024-08-08T20:48:49.6329922Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_int8_weight_fp32_activation_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6330624Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_int8_weight_fp32_activation_native.h 2024-08-08T20:48:49.6331698Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_int8_weight_fp32_activation_ops.h 2024-08-08T20:48:49.6332594Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_int8_weight_native.h 2024-08-08T20:48:49.6333590Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_int8_weight_ops.h 2024-08-08T20:48:49.6334685Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_quantize_weight.h 2024-08-08T20:48:49.6335922Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_quantize_weight_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6336670Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_quantize_weight_native.h 2024-08-08T20:48:49.6337709Z inflating: build/lib/torch/include/ATen/ops/fbgemm_linear_quantize_weight_ops.h 2024-08-08T20:48:49.6338613Z inflating: build/lib/torch/include/ATen/ops/fbgemm_pack_gemm_matrix_fp16.h 2024-08-08T20:48:49.6339854Z inflating: build/lib/torch/include/ATen/ops/fbgemm_pack_gemm_matrix_fp16_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6340699Z inflating: build/lib/torch/include/ATen/ops/fbgemm_pack_gemm_matrix_fp16_native.h 2024-08-08T20:48:49.6341688Z inflating: build/lib/torch/include/ATen/ops/fbgemm_pack_gemm_matrix_fp16_ops.h 2024-08-08T20:48:49.6342619Z inflating: build/lib/torch/include/ATen/ops/fbgemm_pack_quantized_matrix.h 2024-08-08T20:48:49.6343875Z inflating: build/lib/torch/include/ATen/ops/fbgemm_pack_quantized_matrix_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6344656Z inflating: build/lib/torch/include/ATen/ops/fbgemm_pack_quantized_matrix_native.h 2024-08-08T20:48:49.6345673Z inflating: build/lib/torch/include/ATen/ops/fbgemm_pack_quantized_matrix_ops.h 2024-08-08T20:48:49.6346536Z inflating: build/lib/torch/include/ATen/ops/feature_alpha_dropout.h 2024-08-08T20:48:49.6347833Z inflating: build/lib/torch/include/ATen/ops/feature_alpha_dropout_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6348578Z inflating: build/lib/torch/include/ATen/ops/feature_alpha_dropout_native.h 2024-08-08T20:48:49.6349795Z inflating: build/lib/torch/include/ATen/ops/feature_alpha_dropout_ops.h 2024-08-08T20:48:49.6351468Z inflating: build/lib/torch/include/ATen/ops/feature_dropout.h 2024-08-08T20:48:49.6352431Z inflating: build/lib/torch/include/ATen/ops/feature_dropout_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6353260Z inflating: build/lib/torch/include/ATen/ops/feature_dropout_native.h 2024-08-08T20:48:49.6354317Z inflating: build/lib/torch/include/ATen/ops/feature_dropout_ops.h 2024-08-08T20:48:49.6355736Z inflating: build/lib/torch/include/ATen/ops/fft_fft.h 2024-08-08T20:48:49.6356702Z inflating: build/lib/torch/include/ATen/ops/fft_fft2.h 2024-08-08T20:48:49.6358044Z inflating: build/lib/torch/include/ATen/ops/fft_fft2_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6359013Z inflating: build/lib/torch/include/ATen/ops/fft_fft2_native.h 2024-08-08T20:48:49.6360094Z inflating: build/lib/torch/include/ATen/ops/fft_fft2_ops.h 2024-08-08T20:48:49.6361422Z inflating: build/lib/torch/include/ATen/ops/fft_fft_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6362146Z inflating: build/lib/torch/include/ATen/ops/fft_fft_native.h 2024-08-08T20:48:49.6363214Z inflating: build/lib/torch/include/ATen/ops/fft_fft_ops.h 2024-08-08T20:48:49.6364215Z inflating: build/lib/torch/include/ATen/ops/fft_fftfreq.h 2024-08-08T20:48:49.6365555Z inflating: build/lib/torch/include/ATen/ops/fft_fftfreq_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6366286Z inflating: build/lib/torch/include/ATen/ops/fft_fftfreq_native.h 2024-08-08T20:48:49.6367362Z inflating: build/lib/torch/include/ATen/ops/fft_fftfreq_ops.h 2024-08-08T20:48:49.6368806Z inflating: build/lib/torch/include/ATen/ops/fft_fftn.h 2024-08-08T20:48:49.6369919Z inflating: build/lib/torch/include/ATen/ops/fft_fftn_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6370759Z inflating: build/lib/torch/include/ATen/ops/fft_fftn_native.h 2024-08-08T20:48:49.6371784Z inflating: build/lib/torch/include/ATen/ops/fft_fftn_ops.h 2024-08-08T20:48:49.6373221Z inflating: build/lib/torch/include/ATen/ops/fft_fftshift.h 2024-08-08T20:48:49.6374168Z inflating: build/lib/torch/include/ATen/ops/fft_fftshift_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6374941Z inflating: build/lib/torch/include/ATen/ops/fft_fftshift_native.h 2024-08-08T20:48:49.6375941Z inflating: build/lib/torch/include/ATen/ops/fft_fftshift_ops.h 2024-08-08T20:48:49.6377395Z inflating: build/lib/torch/include/ATen/ops/fft_hfft.h 2024-08-08T20:48:49.6378371Z inflating: build/lib/torch/include/ATen/ops/fft_hfft2.h 2024-08-08T20:48:49.6379880Z inflating: build/lib/torch/include/ATen/ops/fft_hfft2_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6380497Z inflating: build/lib/torch/include/ATen/ops/fft_hfft2_native.h 2024-08-08T20:48:49.6381571Z inflating: build/lib/torch/include/ATen/ops/fft_hfft2_ops.h 2024-08-08T20:48:49.6382988Z inflating: build/lib/torch/include/ATen/ops/fft_hfft_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6383619Z inflating: build/lib/torch/include/ATen/ops/fft_hfft_native.h 2024-08-08T20:48:49.6385146Z inflating: build/lib/torch/include/ATen/ops/fft_hfft_ops.h 2024-08-08T20:48:49.6386135Z inflating: build/lib/torch/include/ATen/ops/fft_hfftn.h 2024-08-08T20:48:49.6387418Z inflating: build/lib/torch/include/ATen/ops/fft_hfftn_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6388162Z inflating: build/lib/torch/include/ATen/ops/fft_hfftn_native.h 2024-08-08T20:48:49.6389216Z inflating: build/lib/torch/include/ATen/ops/fft_hfftn_ops.h 2024-08-08T20:48:49.6390605Z inflating: build/lib/torch/include/ATen/ops/fft_ifft.h 2024-08-08T20:48:49.6391555Z inflating: build/lib/torch/include/ATen/ops/fft_ifft2.h 2024-08-08T20:48:49.6392908Z inflating: build/lib/torch/include/ATen/ops/fft_ifft2_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6393637Z inflating: build/lib/torch/include/ATen/ops/fft_ifft2_native.h 2024-08-08T20:48:49.6394687Z inflating: build/lib/torch/include/ATen/ops/fft_ifft2_ops.h 2024-08-08T20:48:49.6395958Z inflating: build/lib/torch/include/ATen/ops/fft_ifft_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6396704Z inflating: build/lib/torch/include/ATen/ops/fft_ifft_native.h 2024-08-08T20:48:49.6397808Z inflating: build/lib/torch/include/ATen/ops/fft_ifft_ops.h 2024-08-08T20:48:49.6399429Z inflating: build/lib/torch/include/ATen/ops/fft_ifftn.h 2024-08-08T20:48:49.6400534Z inflating: build/lib/torch/include/ATen/ops/fft_ifftn_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6401208Z inflating: build/lib/torch/include/ATen/ops/fft_ifftn_native.h 2024-08-08T20:48:49.6402253Z inflating: build/lib/torch/include/ATen/ops/fft_ifftn_ops.h 2024-08-08T20:48:49.6403208Z inflating: build/lib/torch/include/ATen/ops/fft_ifftshift.h 2024-08-08T20:48:49.6404429Z inflating: build/lib/torch/include/ATen/ops/fft_ifftshift_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6405145Z inflating: build/lib/torch/include/ATen/ops/fft_ifftshift_native.h 2024-08-08T20:48:49.6406147Z inflating: build/lib/torch/include/ATen/ops/fft_ifftshift_ops.h 2024-08-08T20:48:49.6407363Z inflating: build/lib/torch/include/ATen/ops/fft_ihfft.h 2024-08-08T20:48:49.6408590Z inflating: build/lib/torch/include/ATen/ops/fft_ihfft2.h 2024-08-08T20:48:49.6409853Z inflating: build/lib/torch/include/ATen/ops/fft_ihfft2_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6410597Z inflating: build/lib/torch/include/ATen/ops/fft_ihfft2_native.h 2024-08-08T20:48:49.6411639Z inflating: build/lib/torch/include/ATen/ops/fft_ihfft2_ops.h 2024-08-08T20:48:49.6412900Z inflating: build/lib/torch/include/ATen/ops/fft_ihfft_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6413657Z inflating: build/lib/torch/include/ATen/ops/fft_ihfft_native.h 2024-08-08T20:48:49.6414668Z inflating: build/lib/torch/include/ATen/ops/fft_ihfft_ops.h 2024-08-08T20:48:49.6416106Z inflating: build/lib/torch/include/ATen/ops/fft_ihfftn.h 2024-08-08T20:48:49.6417139Z inflating: build/lib/torch/include/ATen/ops/fft_ihfftn_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6417934Z inflating: build/lib/torch/include/ATen/ops/fft_ihfftn_native.h 2024-08-08T20:48:49.6419457Z inflating: build/lib/torch/include/ATen/ops/fft_ihfftn_ops.h 2024-08-08T20:48:49.6421303Z inflating: build/lib/torch/include/ATen/ops/fft_irfft.h 2024-08-08T20:48:49.6422340Z inflating: build/lib/torch/include/ATen/ops/fft_irfft2.h 2024-08-08T20:48:49.6423860Z inflating: build/lib/torch/include/ATen/ops/fft_irfft2_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6424506Z inflating: build/lib/torch/include/ATen/ops/fft_irfft2_native.h 2024-08-08T20:48:49.6425641Z inflating: build/lib/torch/include/ATen/ops/fft_irfft2_ops.h 2024-08-08T20:48:49.6426905Z inflating: build/lib/torch/include/ATen/ops/fft_irfft_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6427827Z inflating: build/lib/torch/include/ATen/ops/fft_irfft_native.h 2024-08-08T20:48:49.6428719Z inflating: build/lib/torch/include/ATen/ops/fft_irfft_ops.h 2024-08-08T20:48:49.6430147Z inflating: build/lib/torch/include/ATen/ops/fft_irfftn.h 2024-08-08T20:48:49.6431312Z inflating: build/lib/torch/include/ATen/ops/fft_irfftn_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6432083Z inflating: build/lib/torch/include/ATen/ops/fft_irfftn_native.h 2024-08-08T20:48:49.6433057Z inflating: build/lib/torch/include/ATen/ops/fft_irfftn_ops.h 2024-08-08T20:48:49.6434442Z inflating: build/lib/torch/include/ATen/ops/fft_rfft.h 2024-08-08T20:48:49.6435435Z inflating: build/lib/torch/include/ATen/ops/fft_rfft2.h 2024-08-08T20:48:49.6436818Z inflating: build/lib/torch/include/ATen/ops/fft_rfft2_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6437550Z inflating: build/lib/torch/include/ATen/ops/fft_rfft2_native.h 2024-08-08T20:48:49.6438612Z inflating: build/lib/torch/include/ATen/ops/fft_rfft2_ops.h 2024-08-08T20:48:49.6439875Z inflating: build/lib/torch/include/ATen/ops/fft_rfft_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6440620Z inflating: build/lib/torch/include/ATen/ops/fft_rfft_native.h 2024-08-08T20:48:49.6441673Z inflating: build/lib/torch/include/ATen/ops/fft_rfft_ops.h 2024-08-08T20:48:49.6442684Z inflating: build/lib/torch/include/ATen/ops/fft_rfftfreq.h 2024-08-08T20:48:49.6444076Z inflating: build/lib/torch/include/ATen/ops/fft_rfftfreq_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6444817Z inflating: build/lib/torch/include/ATen/ops/fft_rfftfreq_native.h 2024-08-08T20:48:49.6445866Z inflating: build/lib/torch/include/ATen/ops/fft_rfftfreq_ops.h 2024-08-08T20:48:49.6447298Z inflating: build/lib/torch/include/ATen/ops/fft_rfftn.h 2024-08-08T20:48:49.6448423Z inflating: build/lib/torch/include/ATen/ops/fft_rfftn_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6449154Z inflating: build/lib/torch/include/ATen/ops/fft_rfftn_native.h 2024-08-08T20:48:49.6450232Z inflating: build/lib/torch/include/ATen/ops/fft_rfftn_ops.h 2024-08-08T20:48:49.6451189Z inflating: build/lib/torch/include/ATen/ops/fill.h 2024-08-08T20:48:49.6452676Z inflating: build/lib/torch/include/ATen/ops/fill_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6453373Z inflating: build/lib/torch/include/ATen/ops/fill_cpu_dispatch.h 2024-08-08T20:48:49.6454523Z inflating: build/lib/torch/include/ATen/ops/fill_cuda_dispatch.h 2024-08-08T20:48:49.6455425Z inflating: build/lib/torch/include/ATen/ops/fill_diagonal.h 2024-08-08T20:48:49.6456673Z inflating: build/lib/torch/include/ATen/ops/fill_diagonal_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6457363Z inflating: build/lib/torch/include/ATen/ops/fill_diagonal_native.h 2024-08-08T20:48:49.6458590Z inflating: build/lib/torch/include/ATen/ops/fill_diagonal_ops.h 2024-08-08T20:48:49.6459674Z inflating: build/lib/torch/include/ATen/ops/fill_meta_dispatch.h 2024-08-08T20:48:49.6460619Z inflating: build/lib/torch/include/ATen/ops/fill_native.h 2024-08-08T20:48:49.6462003Z inflating: build/lib/torch/include/ATen/ops/fill_ops.h 2024-08-08T20:48:49.6462743Z inflating: build/lib/torch/include/ATen/ops/fix.h 2024-08-08T20:48:49.6464098Z inflating: build/lib/torch/include/ATen/ops/fix_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6464754Z inflating: build/lib/torch/include/ATen/ops/fix_native.h 2024-08-08T20:48:49.6465831Z inflating: build/lib/torch/include/ATen/ops/fix_ops.h 2024-08-08T20:48:49.6466862Z inflating: build/lib/torch/include/ATen/ops/flatten.h 2024-08-08T20:48:49.6468194Z inflating: build/lib/torch/include/ATen/ops/flatten_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6469005Z inflating: build/lib/torch/include/ATen/ops/flatten_dense_tensors.h 2024-08-08T20:48:49.6470537Z inflating: build/lib/torch/include/ATen/ops/flatten_dense_tensors_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6471158Z inflating: build/lib/torch/include/ATen/ops/flatten_dense_tensors_native.h 2024-08-08T20:48:49.6472078Z inflating: build/lib/torch/include/ATen/ops/flatten_dense_tensors_ops.h 2024-08-08T20:48:49.6472890Z inflating: build/lib/torch/include/ATen/ops/flatten_native.h 2024-08-08T20:48:49.6474003Z inflating: build/lib/torch/include/ATen/ops/flatten_ops.h 2024-08-08T20:48:49.6474916Z inflating: build/lib/torch/include/ATen/ops/flip.h 2024-08-08T20:48:49.6476192Z inflating: build/lib/torch/include/ATen/ops/flip_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6477001Z inflating: build/lib/torch/include/ATen/ops/flip_cpu_dispatch.h 2024-08-08T20:48:49.6477990Z inflating: build/lib/torch/include/ATen/ops/flip_cuda_dispatch.h 2024-08-08T20:48:49.6478880Z inflating: build/lib/torch/include/ATen/ops/flip_native.h 2024-08-08T20:48:49.6479958Z inflating: build/lib/torch/include/ATen/ops/flip_ops.h 2024-08-08T20:48:49.6480757Z inflating: build/lib/torch/include/ATen/ops/fliplr.h 2024-08-08T20:48:49.6482278Z inflating: build/lib/torch/include/ATen/ops/fliplr_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6482826Z inflating: build/lib/torch/include/ATen/ops/fliplr_native.h 2024-08-08T20:48:49.6483815Z inflating: build/lib/torch/include/ATen/ops/fliplr_ops.h 2024-08-08T20:48:49.6484700Z inflating: build/lib/torch/include/ATen/ops/flipud.h 2024-08-08T20:48:49.6485964Z inflating: build/lib/torch/include/ATen/ops/flipud_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6486856Z inflating: build/lib/torch/include/ATen/ops/flipud_native.h 2024-08-08T20:48:49.6488662Z inflating: build/lib/torch/include/ATen/ops/flipud_ops.h 2024-08-08T20:48:49.6489446Z inflating: build/lib/torch/include/ATen/ops/float_power.h 2024-08-08T20:48:49.6490836Z inflating: build/lib/torch/include/ATen/ops/float_power_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6491510Z inflating: build/lib/torch/include/ATen/ops/float_power_native.h 2024-08-08T20:48:49.6493121Z inflating: build/lib/torch/include/ATen/ops/float_power_ops.h 2024-08-08T20:48:49.6493702Z inflating: build/lib/torch/include/ATen/ops/floor.h 2024-08-08T20:48:49.6495461Z inflating: build/lib/torch/include/ATen/ops/floor_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6496057Z inflating: build/lib/torch/include/ATen/ops/floor_cpu_dispatch.h 2024-08-08T20:48:49.6496984Z inflating: build/lib/torch/include/ATen/ops/floor_cuda_dispatch.h 2024-08-08T20:48:49.6497966Z inflating: build/lib/torch/include/ATen/ops/floor_divide.h 2024-08-08T20:48:49.6499234Z inflating: build/lib/torch/include/ATen/ops/floor_divide_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6500098Z inflating: build/lib/torch/include/ATen/ops/floor_divide_cpu_dispatch.h 2024-08-08T20:48:49.6501143Z inflating: build/lib/torch/include/ATen/ops/floor_divide_cuda_dispatch.h 2024-08-08T20:48:49.6502258Z inflating: build/lib/torch/include/ATen/ops/floor_divide_meta_dispatch.h 2024-08-08T20:48:49.6503003Z inflating: build/lib/torch/include/ATen/ops/floor_divide_native.h 2024-08-08T20:48:49.6504104Z inflating: build/lib/torch/include/ATen/ops/floor_divide_ops.h 2024-08-08T20:48:49.6505087Z inflating: build/lib/torch/include/ATen/ops/floor_meta.h 2024-08-08T20:48:49.6506130Z inflating: build/lib/torch/include/ATen/ops/floor_meta_dispatch.h 2024-08-08T20:48:49.6507085Z inflating: build/lib/torch/include/ATen/ops/floor_native.h 2024-08-08T20:48:49.6508102Z inflating: build/lib/torch/include/ATen/ops/floor_ops.h 2024-08-08T20:48:49.6509052Z inflating: build/lib/torch/include/ATen/ops/fmax.h 2024-08-08T20:48:49.6510447Z inflating: build/lib/torch/include/ATen/ops/fmax_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6511165Z inflating: build/lib/torch/include/ATen/ops/fmax_cpu_dispatch.h 2024-08-08T20:48:49.6512151Z inflating: build/lib/torch/include/ATen/ops/fmax_cuda_dispatch.h 2024-08-08T20:48:49.6513263Z inflating: build/lib/torch/include/ATen/ops/fmax_meta.h 2024-08-08T20:48:49.6514185Z inflating: build/lib/torch/include/ATen/ops/fmax_meta_dispatch.h 2024-08-08T20:48:49.6515050Z inflating: build/lib/torch/include/ATen/ops/fmax_native.h 2024-08-08T20:48:49.6516438Z inflating: build/lib/torch/include/ATen/ops/fmax_ops.h 2024-08-08T20:48:49.6516970Z inflating: build/lib/torch/include/ATen/ops/fmin.h 2024-08-08T20:48:49.6518765Z inflating: build/lib/torch/include/ATen/ops/fmin_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6519224Z inflating: build/lib/torch/include/ATen/ops/fmin_cpu_dispatch.h 2024-08-08T20:48:49.6520335Z inflating: build/lib/torch/include/ATen/ops/fmin_cuda_dispatch.h 2024-08-08T20:48:49.6521254Z inflating: build/lib/torch/include/ATen/ops/fmin_meta.h 2024-08-08T20:48:49.6522279Z inflating: build/lib/torch/include/ATen/ops/fmin_meta_dispatch.h 2024-08-08T20:48:49.6523190Z inflating: build/lib/torch/include/ATen/ops/fmin_native.h 2024-08-08T20:48:49.6524220Z inflating: build/lib/torch/include/ATen/ops/fmin_ops.h 2024-08-08T20:48:49.6525149Z inflating: build/lib/torch/include/ATen/ops/fmod.h 2024-08-08T20:48:49.6526509Z inflating: build/lib/torch/include/ATen/ops/fmod_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6527603Z inflating: build/lib/torch/include/ATen/ops/fmod_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6528327Z inflating: build/lib/torch/include/ATen/ops/fmod_cpu_dispatch.h 2024-08-08T20:48:49.6529304Z inflating: build/lib/torch/include/ATen/ops/fmod_cuda_dispatch.h 2024-08-08T20:48:49.6530164Z inflating: build/lib/torch/include/ATen/ops/fmod_meta.h 2024-08-08T20:48:49.6531235Z inflating: build/lib/torch/include/ATen/ops/fmod_meta_dispatch.h 2024-08-08T20:48:49.6532175Z inflating: build/lib/torch/include/ATen/ops/fmod_native.h 2024-08-08T20:48:49.6533753Z inflating: build/lib/torch/include/ATen/ops/fmod_ops.h 2024-08-08T20:48:49.6534267Z inflating: build/lib/torch/include/ATen/ops/frac.h 2024-08-08T20:48:49.6535712Z inflating: build/lib/torch/include/ATen/ops/frac_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6536405Z inflating: build/lib/torch/include/ATen/ops/frac_cpu_dispatch.h 2024-08-08T20:48:49.6537361Z inflating: build/lib/torch/include/ATen/ops/frac_cuda_dispatch.h 2024-08-08T20:48:49.6538270Z inflating: build/lib/torch/include/ATen/ops/frac_meta.h 2024-08-08T20:48:49.6539276Z inflating: build/lib/torch/include/ATen/ops/frac_meta_dispatch.h 2024-08-08T20:48:49.6540232Z inflating: build/lib/torch/include/ATen/ops/frac_native.h 2024-08-08T20:48:49.6541283Z inflating: build/lib/torch/include/ATen/ops/frac_ops.h 2024-08-08T20:48:49.6542422Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d.h 2024-08-08T20:48:49.6543623Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_backward.h 2024-08-08T20:48:49.6544954Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6545735Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.6546752Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.6547727Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_backward_meta.h 2024-08-08T20:48:49.6548759Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_backward_meta_dispatch.h 2024-08-08T20:48:49.6549710Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_backward_native.h 2024-08-08T20:48:49.6550826Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_backward_ops.h 2024-08-08T20:48:49.6552097Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6553106Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_cpu_dispatch.h 2024-08-08T20:48:49.6554934Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_cuda_dispatch.h 2024-08-08T20:48:49.6555603Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_meta.h 2024-08-08T20:48:49.6556838Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_meta_dispatch.h 2024-08-08T20:48:49.6557777Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_native.h 2024-08-08T20:48:49.6558988Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool2d_ops.h 2024-08-08T20:48:49.6560060Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d.h 2024-08-08T20:48:49.6561199Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_backward.h 2024-08-08T20:48:49.6562304Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_backward_cpu_dispatch.h 2024-08-08T20:48:49.6563334Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_backward_cuda_dispatch.h 2024-08-08T20:48:49.6564246Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_backward_native.h 2024-08-08T20:48:49.6565355Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_backward_ops.h 2024-08-08T20:48:49.6566624Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6567390Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_cpu_dispatch.h 2024-08-08T20:48:49.6568411Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_cuda_dispatch.h 2024-08-08T20:48:49.6570133Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_meta.h 2024-08-08T20:48:49.6571011Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_meta_dispatch.h 2024-08-08T20:48:49.6572013Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_native.h 2024-08-08T20:48:49.6573063Z inflating: build/lib/torch/include/ATen/ops/fractional_max_pool3d_ops.h 2024-08-08T20:48:49.6573859Z inflating: build/lib/torch/include/ATen/ops/frexp.h 2024-08-08T20:48:49.6575140Z inflating: build/lib/torch/include/ATen/ops/frexp_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6575951Z inflating: build/lib/torch/include/ATen/ops/frexp_cpu_dispatch.h 2024-08-08T20:48:49.6576948Z inflating: build/lib/torch/include/ATen/ops/frexp_cuda_dispatch.h 2024-08-08T20:48:49.6577833Z inflating: build/lib/torch/include/ATen/ops/frexp_native.h 2024-08-08T20:48:49.6578894Z inflating: build/lib/torch/include/ATen/ops/frexp_ops.h 2024-08-08T20:48:49.6579952Z inflating: build/lib/torch/include/ATen/ops/frobenius_norm.h 2024-08-08T20:48:49.6581314Z inflating: build/lib/torch/include/ATen/ops/frobenius_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6582161Z inflating: build/lib/torch/include/ATen/ops/frobenius_norm_native.h 2024-08-08T20:48:49.6583120Z inflating: build/lib/torch/include/ATen/ops/frobenius_norm_ops.h 2024-08-08T20:48:49.6584136Z inflating: build/lib/torch/include/ATen/ops/from_file.h 2024-08-08T20:48:49.6585453Z inflating: build/lib/torch/include/ATen/ops/from_file_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6586292Z inflating: build/lib/torch/include/ATen/ops/from_file_cpu_dispatch.h 2024-08-08T20:48:49.6587408Z inflating: build/lib/torch/include/ATen/ops/from_file_native.h 2024-08-08T20:48:49.6588540Z inflating: build/lib/torch/include/ATen/ops/from_file_ops.h 2024-08-08T20:48:49.6590047Z inflating: build/lib/torch/include/ATen/ops/full.h 2024-08-08T20:48:49.6591208Z inflating: build/lib/torch/include/ATen/ops/full_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6592012Z inflating: build/lib/torch/include/ATen/ops/full_like.h 2024-08-08T20:48:49.6593363Z inflating: build/lib/torch/include/ATen/ops/full_like_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6594295Z inflating: build/lib/torch/include/ATen/ops/full_like_native.h 2024-08-08T20:48:49.6595953Z inflating: build/lib/torch/include/ATen/ops/full_like_ops.h 2024-08-08T20:48:49.6596272Z inflating: build/lib/torch/include/ATen/ops/full_native.h 2024-08-08T20:48:49.6597891Z inflating: build/lib/torch/include/ATen/ops/full_ops.h 2024-08-08T20:48:49.6598889Z inflating: build/lib/torch/include/ATen/ops/fused_moving_avg_obs_fake_quant.h 2024-08-08T20:48:49.6600184Z inflating: build/lib/torch/include/ATen/ops/fused_moving_avg_obs_fake_quant_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6600944Z inflating: build/lib/torch/include/ATen/ops/fused_moving_avg_obs_fake_quant_native.h 2024-08-08T20:48:49.6602093Z inflating: build/lib/torch/include/ATen/ops/fused_moving_avg_obs_fake_quant_ops.h 2024-08-08T20:48:49.6602901Z inflating: build/lib/torch/include/ATen/ops/gather.h 2024-08-08T20:48:49.6603964Z inflating: build/lib/torch/include/ATen/ops/gather_backward.h 2024-08-08T20:48:49.6605234Z inflating: build/lib/torch/include/ATen/ops/gather_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6605944Z inflating: build/lib/torch/include/ATen/ops/gather_backward_native.h 2024-08-08T20:48:49.6606981Z inflating: build/lib/torch/include/ATen/ops/gather_backward_ops.h 2024-08-08T20:48:49.6608280Z inflating: build/lib/torch/include/ATen/ops/gather_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6609246Z inflating: build/lib/torch/include/ATen/ops/gather_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6610040Z inflating: build/lib/torch/include/ATen/ops/gather_cpu_dispatch.h 2024-08-08T20:48:49.6611030Z inflating: build/lib/torch/include/ATen/ops/gather_cuda_dispatch.h 2024-08-08T20:48:49.6611930Z inflating: build/lib/torch/include/ATen/ops/gather_meta.h 2024-08-08T20:48:49.6613085Z inflating: build/lib/torch/include/ATen/ops/gather_meta_dispatch.h 2024-08-08T20:48:49.6613954Z inflating: build/lib/torch/include/ATen/ops/gather_native.h 2024-08-08T20:48:49.6615029Z inflating: build/lib/torch/include/ATen/ops/gather_ops.h 2024-08-08T20:48:49.6615987Z inflating: build/lib/torch/include/ATen/ops/gcd.h 2024-08-08T20:48:49.6617383Z inflating: build/lib/torch/include/ATen/ops/gcd_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6618097Z inflating: build/lib/torch/include/ATen/ops/gcd_cpu_dispatch.h 2024-08-08T20:48:49.6619105Z inflating: build/lib/torch/include/ATen/ops/gcd_cuda_dispatch.h 2024-08-08T20:48:49.6619963Z inflating: build/lib/torch/include/ATen/ops/gcd_meta.h 2024-08-08T20:48:49.6621503Z inflating: build/lib/torch/include/ATen/ops/gcd_meta_dispatch.h 2024-08-08T20:48:49.6622901Z inflating: build/lib/torch/include/ATen/ops/gcd_native.h 2024-08-08T20:48:49.6623775Z inflating: build/lib/torch/include/ATen/ops/gcd_ops.h 2024-08-08T20:48:49.6624754Z inflating: build/lib/torch/include/ATen/ops/ge.h 2024-08-08T20:48:49.6626254Z inflating: build/lib/torch/include/ATen/ops/ge_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6626972Z inflating: build/lib/torch/include/ATen/ops/ge_cpu_dispatch.h 2024-08-08T20:48:49.6627952Z inflating: build/lib/torch/include/ATen/ops/ge_cuda_dispatch.h 2024-08-08T20:48:49.6628833Z inflating: build/lib/torch/include/ATen/ops/ge_meta.h 2024-08-08T20:48:49.6629911Z inflating: build/lib/torch/include/ATen/ops/ge_meta_dispatch.h 2024-08-08T20:48:49.6630903Z inflating: build/lib/torch/include/ATen/ops/ge_native.h 2024-08-08T20:48:49.6632363Z inflating: build/lib/torch/include/ATen/ops/ge_ops.h 2024-08-08T20:48:49.6632984Z inflating: build/lib/torch/include/ATen/ops/gelu.h 2024-08-08T20:48:49.6634424Z inflating: build/lib/torch/include/ATen/ops/gelu_backward.h 2024-08-08T20:48:49.6635435Z inflating: build/lib/torch/include/ATen/ops/gelu_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6636395Z inflating: build/lib/torch/include/ATen/ops/gelu_backward_cpu_dispatch.h 2024-08-08T20:48:49.6637380Z inflating: build/lib/torch/include/ATen/ops/gelu_backward_cuda_dispatch.h 2024-08-08T20:48:49.6638157Z inflating: build/lib/torch/include/ATen/ops/gelu_backward_meta.h 2024-08-08T20:48:49.6639306Z inflating: build/lib/torch/include/ATen/ops/gelu_backward_meta_dispatch.h 2024-08-08T20:48:49.6640281Z inflating: build/lib/torch/include/ATen/ops/gelu_backward_native.h 2024-08-08T20:48:49.6641298Z inflating: build/lib/torch/include/ATen/ops/gelu_backward_ops.h 2024-08-08T20:48:49.6642619Z inflating: build/lib/torch/include/ATen/ops/gelu_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6643294Z inflating: build/lib/torch/include/ATen/ops/gelu_cpu_dispatch.h 2024-08-08T20:48:49.6644297Z inflating: build/lib/torch/include/ATen/ops/gelu_cuda_dispatch.h 2024-08-08T20:48:49.6645262Z inflating: build/lib/torch/include/ATen/ops/gelu_meta.h 2024-08-08T20:48:49.6646348Z inflating: build/lib/torch/include/ATen/ops/gelu_meta_dispatch.h 2024-08-08T20:48:49.6647331Z inflating: build/lib/torch/include/ATen/ops/gelu_native.h 2024-08-08T20:48:49.6648340Z inflating: build/lib/torch/include/ATen/ops/gelu_ops.h 2024-08-08T20:48:49.6649356Z inflating: build/lib/torch/include/ATen/ops/geometric.h 2024-08-08T20:48:49.6650702Z inflating: build/lib/torch/include/ATen/ops/geometric_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6651554Z inflating: build/lib/torch/include/ATen/ops/geometric_cpu_dispatch.h 2024-08-08T20:48:49.6652530Z inflating: build/lib/torch/include/ATen/ops/geometric_cuda_dispatch.h 2024-08-08T20:48:49.6653515Z inflating: build/lib/torch/include/ATen/ops/geometric_meta_dispatch.h 2024-08-08T20:48:49.6654595Z inflating: build/lib/torch/include/ATen/ops/geometric_native.h 2024-08-08T20:48:49.6655691Z inflating: build/lib/torch/include/ATen/ops/geometric_ops.h 2024-08-08T20:48:49.6656595Z inflating: build/lib/torch/include/ATen/ops/geqrf.h 2024-08-08T20:48:49.6657726Z inflating: build/lib/torch/include/ATen/ops/geqrf_cpu_dispatch.h 2024-08-08T20:48:49.6658902Z inflating: build/lib/torch/include/ATen/ops/geqrf_cuda_dispatch.h 2024-08-08T20:48:49.6672748Z inflating: build/lib/torch/include/ATen/ops/geqrf_native.h 2024-08-08T20:48:49.6673428Z inflating: build/lib/torch/include/ATen/ops/geqrf_ops.h 2024-08-08T20:48:49.6673880Z inflating: build/lib/torch/include/ATen/ops/ger.h 2024-08-08T20:48:49.6674451Z inflating: build/lib/torch/include/ATen/ops/ger_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6674742Z inflating: build/lib/torch/include/ATen/ops/ger_native.h 2024-08-08T20:48:49.6675054Z inflating: build/lib/torch/include/ATen/ops/ger_ops.h 2024-08-08T20:48:49.6675230Z inflating: build/lib/torch/include/ATen/ops/glu.h 2024-08-08T20:48:49.6675875Z inflating: build/lib/torch/include/ATen/ops/glu_backward.h 2024-08-08T20:48:49.6676264Z inflating: build/lib/torch/include/ATen/ops/glu_backward_cpu_dispatch.h 2024-08-08T20:48:49.6676750Z inflating: build/lib/torch/include/ATen/ops/glu_backward_cuda_dispatch.h 2024-08-08T20:48:49.6677090Z inflating: build/lib/torch/include/ATen/ops/glu_backward_jvp.h 2024-08-08T20:48:49.6677729Z inflating: build/lib/torch/include/ATen/ops/glu_backward_jvp_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6678188Z inflating: build/lib/torch/include/ATen/ops/glu_backward_jvp_cpu_dispatch.h 2024-08-08T20:48:49.6678630Z inflating: build/lib/torch/include/ATen/ops/glu_backward_jvp_cuda_dispatch.h 2024-08-08T20:48:49.6679048Z inflating: build/lib/torch/include/ATen/ops/glu_backward_jvp_native.h 2024-08-08T20:48:49.6679410Z inflating: build/lib/torch/include/ATen/ops/glu_backward_jvp_ops.h 2024-08-08T20:48:49.6679778Z inflating: build/lib/torch/include/ATen/ops/glu_backward_native.h 2024-08-08T20:48:49.6680159Z inflating: build/lib/torch/include/ATen/ops/glu_backward_ops.h 2024-08-08T20:48:49.6680942Z inflating: build/lib/torch/include/ATen/ops/glu_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6681315Z inflating: build/lib/torch/include/ATen/ops/glu_cpu_dispatch.h 2024-08-08T20:48:49.6681677Z inflating: build/lib/torch/include/ATen/ops/glu_cuda_dispatch.h 2024-08-08T20:48:49.6681943Z inflating: build/lib/torch/include/ATen/ops/glu_jvp.h 2024-08-08T20:48:49.6682515Z inflating: build/lib/torch/include/ATen/ops/glu_jvp_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6682900Z inflating: build/lib/torch/include/ATen/ops/glu_jvp_cpu_dispatch.h 2024-08-08T20:48:49.6683235Z inflating: build/lib/torch/include/ATen/ops/glu_jvp_cuda_dispatch.h 2024-08-08T20:48:49.6683613Z inflating: build/lib/torch/include/ATen/ops/glu_jvp_native.h 2024-08-08T20:48:49.6683921Z inflating: build/lib/torch/include/ATen/ops/glu_jvp_ops.h 2024-08-08T20:48:49.6684822Z inflating: build/lib/torch/include/ATen/ops/glu_meta.h 2024-08-08T20:48:49.6686637Z inflating: build/lib/torch/include/ATen/ops/glu_meta_dispatch.h 2024-08-08T20:48:49.6687221Z inflating: build/lib/torch/include/ATen/ops/glu_native.h 2024-08-08T20:48:49.6688311Z inflating: build/lib/torch/include/ATen/ops/glu_ops.h 2024-08-08T20:48:49.6689742Z inflating: build/lib/torch/include/ATen/ops/gradient.h 2024-08-08T20:48:49.6690712Z inflating: build/lib/torch/include/ATen/ops/gradient_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6691545Z inflating: build/lib/torch/include/ATen/ops/gradient_native.h 2024-08-08T20:48:49.6693046Z inflating: build/lib/torch/include/ATen/ops/gradient_ops.h 2024-08-08T20:48:49.6693819Z inflating: build/lib/torch/include/ATen/ops/greater.h 2024-08-08T20:48:49.6695405Z inflating: build/lib/torch/include/ATen/ops/greater_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6695930Z inflating: build/lib/torch/include/ATen/ops/greater_equal.h 2024-08-08T20:48:49.6697331Z inflating: build/lib/torch/include/ATen/ops/greater_equal_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6698024Z inflating: build/lib/torch/include/ATen/ops/greater_equal_native.h 2024-08-08T20:48:49.6699518Z inflating: build/lib/torch/include/ATen/ops/greater_equal_ops.h 2024-08-08T20:48:49.6700695Z inflating: build/lib/torch/include/ATen/ops/greater_native.h 2024-08-08T20:48:49.6701847Z inflating: build/lib/torch/include/ATen/ops/greater_ops.h 2024-08-08T20:48:49.6702801Z inflating: build/lib/torch/include/ATen/ops/grid_sampler.h 2024-08-08T20:48:49.6703839Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d.h 2024-08-08T20:48:49.6704970Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_backward.h 2024-08-08T20:48:49.6706500Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6707125Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.6708224Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.6709246Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_backward_native.h 2024-08-08T20:48:49.6710422Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_backward_ops.h 2024-08-08T20:48:49.6711673Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6712565Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_cpu_dispatch.h 2024-08-08T20:48:49.6713407Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_cuda_dispatch.h 2024-08-08T20:48:49.6714406Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_native.h 2024-08-08T20:48:49.6715480Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_2d_ops.h 2024-08-08T20:48:49.6716485Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d.h 2024-08-08T20:48:49.6717659Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_backward.h 2024-08-08T20:48:49.6719219Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6720442Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_backward_cpu_dispatch.h 2024-08-08T20:48:49.6721381Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_backward_cuda_dispatch.h 2024-08-08T20:48:49.6722082Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_backward_native.h 2024-08-08T20:48:49.6723098Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_backward_ops.h 2024-08-08T20:48:49.6724463Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6725183Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_cpu_dispatch.h 2024-08-08T20:48:49.6725768Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_cuda_dispatch.h 2024-08-08T20:48:49.6726830Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_native.h 2024-08-08T20:48:49.6727800Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_3d_ops.h 2024-08-08T20:48:49.6729094Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6729936Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_native.h 2024-08-08T20:48:49.6731218Z inflating: build/lib/torch/include/ATen/ops/grid_sampler_ops.h 2024-08-08T20:48:49.6732184Z inflating: build/lib/torch/include/ATen/ops/group_norm.h 2024-08-08T20:48:49.6733736Z inflating: build/lib/torch/include/ATen/ops/group_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6734270Z inflating: build/lib/torch/include/ATen/ops/group_norm_native.h 2024-08-08T20:48:49.6735258Z inflating: build/lib/torch/include/ATen/ops/group_norm_ops.h 2024-08-08T20:48:49.6736152Z inflating: build/lib/torch/include/ATen/ops/gru.h 2024-08-08T20:48:49.6737202Z inflating: build/lib/torch/include/ATen/ops/gru_cell.h 2024-08-08T20:48:49.6738693Z inflating: build/lib/torch/include/ATen/ops/gru_cell_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6739216Z inflating: build/lib/torch/include/ATen/ops/gru_cell_native.h 2024-08-08T20:48:49.6739922Z inflating: build/lib/torch/include/ATen/ops/gru_cell_ops.h 2024-08-08T20:48:49.6741310Z inflating: build/lib/torch/include/ATen/ops/gru_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6741925Z inflating: build/lib/torch/include/ATen/ops/gru_native.h 2024-08-08T20:48:49.6743342Z inflating: build/lib/torch/include/ATen/ops/gru_ops.h 2024-08-08T20:48:49.6743973Z inflating: build/lib/torch/include/ATen/ops/gt.h 2024-08-08T20:48:49.6745710Z inflating: build/lib/torch/include/ATen/ops/gt_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6746117Z inflating: build/lib/torch/include/ATen/ops/gt_cpu_dispatch.h 2024-08-08T20:48:49.6747202Z inflating: build/lib/torch/include/ATen/ops/gt_cuda_dispatch.h 2024-08-08T20:48:49.6747993Z inflating: build/lib/torch/include/ATen/ops/gt_meta.h 2024-08-08T20:48:49.6749144Z inflating: build/lib/torch/include/ATen/ops/gt_meta_dispatch.h 2024-08-08T20:48:49.6750208Z inflating: build/lib/torch/include/ATen/ops/gt_native.h 2024-08-08T20:48:49.6751471Z inflating: build/lib/torch/include/ATen/ops/gt_ops.h 2024-08-08T20:48:49.6753291Z inflating: build/lib/torch/include/ATen/ops/hamming_window.h 2024-08-08T20:48:49.6755569Z inflating: build/lib/torch/include/ATen/ops/hamming_window_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6756144Z inflating: build/lib/torch/include/ATen/ops/hamming_window_native.h 2024-08-08T20:48:49.6757427Z inflating: build/lib/torch/include/ATen/ops/hamming_window_ops.h 2024-08-08T20:48:49.6758756Z inflating: build/lib/torch/include/ATen/ops/hann_window.h 2024-08-08T20:48:49.6760320Z inflating: build/lib/torch/include/ATen/ops/hann_window_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6760843Z inflating: build/lib/torch/include/ATen/ops/hann_window_native.h 2024-08-08T20:48:49.6761995Z inflating: build/lib/torch/include/ATen/ops/hann_window_ops.h 2024-08-08T20:48:49.6762811Z inflating: build/lib/torch/include/ATen/ops/hardshrink.h 2024-08-08T20:48:49.6764098Z inflating: build/lib/torch/include/ATen/ops/hardshrink_backward.h 2024-08-08T20:48:49.6765806Z inflating: build/lib/torch/include/ATen/ops/hardshrink_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6766683Z inflating: build/lib/torch/include/ATen/ops/hardshrink_backward_cpu_dispatch.h 2024-08-08T20:48:49.6767565Z inflating: build/lib/torch/include/ATen/ops/hardshrink_backward_cuda_dispatch.h 2024-08-08T20:48:49.6768184Z inflating: build/lib/torch/include/ATen/ops/hardshrink_backward_meta.h 2024-08-08T20:48:49.6769365Z inflating: build/lib/torch/include/ATen/ops/hardshrink_backward_meta_dispatch.h 2024-08-08T20:48:49.6770262Z inflating: build/lib/torch/include/ATen/ops/hardshrink_backward_native.h 2024-08-08T20:48:49.6771239Z inflating: build/lib/torch/include/ATen/ops/hardshrink_backward_ops.h 2024-08-08T20:48:49.6772836Z inflating: build/lib/torch/include/ATen/ops/hardshrink_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6773593Z inflating: build/lib/torch/include/ATen/ops/hardshrink_cpu_dispatch.h 2024-08-08T20:48:49.6774163Z inflating: build/lib/torch/include/ATen/ops/hardshrink_cuda_dispatch.h 2024-08-08T20:48:49.6774896Z inflating: build/lib/torch/include/ATen/ops/hardshrink_meta.h 2024-08-08T20:48:49.6776376Z inflating: build/lib/torch/include/ATen/ops/hardshrink_meta_dispatch.h 2024-08-08T20:48:49.6777171Z inflating: build/lib/torch/include/ATen/ops/hardshrink_native.h 2024-08-08T20:48:49.6778216Z inflating: build/lib/torch/include/ATen/ops/hardshrink_ops.h 2024-08-08T20:48:49.6779195Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid.h 2024-08-08T20:48:49.6780278Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_backward.h 2024-08-08T20:48:49.6782150Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6783064Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_backward_cpu_dispatch.h 2024-08-08T20:48:49.6783892Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_backward_cuda_dispatch.h 2024-08-08T20:48:49.6784610Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_backward_meta.h 2024-08-08T20:48:49.6785642Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_backward_meta_dispatch.h 2024-08-08T20:48:49.6786516Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_backward_native.h 2024-08-08T20:48:49.6787686Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_backward_ops.h 2024-08-08T20:48:49.6789118Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6790298Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_cpu_dispatch.h 2024-08-08T20:48:49.6791016Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_cuda_dispatch.h 2024-08-08T20:48:49.6791555Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_meta.h 2024-08-08T20:48:49.6792666Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_meta_dispatch.h 2024-08-08T20:48:49.6793492Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_native.h 2024-08-08T20:48:49.6794587Z inflating: build/lib/torch/include/ATen/ops/hardsigmoid_ops.h 2024-08-08T20:48:49.6795473Z inflating: build/lib/torch/include/ATen/ops/hardswish.h 2024-08-08T20:48:49.6796981Z inflating: build/lib/torch/include/ATen/ops/hardswish_backward.h 2024-08-08T20:48:49.6798523Z inflating: build/lib/torch/include/ATen/ops/hardswish_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6798986Z inflating: build/lib/torch/include/ATen/ops/q_scale_ops.h 2024-08-08T20:48:49.6800263Z inflating: build/lib/torch/include/ATen/ops/hardswish_backward_cpu_dispatch.h 2024-08-08T20:48:49.6801640Z inflating: build/lib/torch/include/ATen/ops/hardswish_backward_cuda_dispatch.h 2024-08-08T20:48:49.6802531Z inflating: build/lib/torch/include/ATen/ops/hardswish_backward_native.h 2024-08-08T20:48:49.6803412Z inflating: build/lib/torch/include/ATen/ops/hardswish_backward_ops.h 2024-08-08T20:48:49.6804410Z inflating: build/lib/torch/include/ATen/ops/hardswish_cpu_dispatch.h 2024-08-08T20:48:49.6805379Z inflating: build/lib/torch/include/ATen/ops/hardswish_cuda_dispatch.h 2024-08-08T20:48:49.6806345Z inflating: build/lib/torch/include/ATen/ops/hardswish_meta_dispatch.h 2024-08-08T20:48:49.6807219Z inflating: build/lib/torch/include/ATen/ops/hardswish_native.h 2024-08-08T20:48:49.6808274Z inflating: build/lib/torch/include/ATen/ops/hardswish_ops.h 2024-08-08T20:48:49.6809237Z inflating: build/lib/torch/include/ATen/ops/hardtanh.h 2024-08-08T20:48:49.6810324Z inflating: build/lib/torch/include/ATen/ops/hardtanh_backward.h 2024-08-08T20:48:49.6811531Z inflating: build/lib/torch/include/ATen/ops/hardtanh_backward_cpu_dispatch.h 2024-08-08T20:48:49.6812539Z inflating: build/lib/torch/include/ATen/ops/hardtanh_backward_cuda_dispatch.h 2024-08-08T20:48:49.6813349Z inflating: build/lib/torch/include/ATen/ops/hardtanh_backward_native.h 2024-08-08T20:48:49.6814441Z inflating: build/lib/torch/include/ATen/ops/hardtanh_backward_ops.h 2024-08-08T20:48:49.6815421Z inflating: build/lib/torch/include/ATen/ops/hardtanh_cpu_dispatch.h 2024-08-08T20:48:49.6816441Z inflating: build/lib/torch/include/ATen/ops/hardtanh_cuda_dispatch.h 2024-08-08T20:48:49.6817370Z inflating: build/lib/torch/include/ATen/ops/hardtanh_meta_dispatch.h 2024-08-08T20:48:49.6818250Z inflating: build/lib/torch/include/ATen/ops/hardtanh_native.h 2024-08-08T20:48:49.6819285Z inflating: build/lib/torch/include/ATen/ops/hardtanh_ops.h 2024-08-08T20:48:49.6820405Z inflating: build/lib/torch/include/ATen/ops/heaviside.h 2024-08-08T20:48:49.6822924Z inflating: build/lib/torch/include/ATen/ops/heaviside_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6823621Z inflating: build/lib/torch/include/ATen/ops/heaviside_cpu_dispatch.h 2024-08-08T20:48:49.6824349Z inflating: build/lib/torch/include/ATen/ops/heaviside_cuda_dispatch.h 2024-08-08T20:48:49.6825112Z inflating: build/lib/torch/include/ATen/ops/heaviside_meta.h 2024-08-08T20:48:49.6826106Z inflating: build/lib/torch/include/ATen/ops/heaviside_meta_dispatch.h 2024-08-08T20:48:49.6827048Z inflating: build/lib/torch/include/ATen/ops/heaviside_native.h 2024-08-08T20:48:49.6828049Z inflating: build/lib/torch/include/ATen/ops/heaviside_ops.h 2024-08-08T20:48:49.6829031Z inflating: build/lib/torch/include/ATen/ops/hinge_embedding_loss.h 2024-08-08T20:48:49.6830544Z inflating: build/lib/torch/include/ATen/ops/hinge_embedding_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6831509Z inflating: build/lib/torch/include/ATen/ops/hinge_embedding_loss_native.h 2024-08-08T20:48:49.6832259Z inflating: build/lib/torch/include/ATen/ops/hinge_embedding_loss_ops.h 2024-08-08T20:48:49.6832959Z inflating: build/lib/torch/include/ATen/ops/histc.h 2024-08-08T20:48:49.6833996Z inflating: build/lib/torch/include/ATen/ops/histc_cpu_dispatch.h 2024-08-08T20:48:49.6834966Z inflating: build/lib/torch/include/ATen/ops/histc_cuda_dispatch.h 2024-08-08T20:48:49.6835851Z inflating: build/lib/torch/include/ATen/ops/histc_native.h 2024-08-08T20:48:49.6836854Z inflating: build/lib/torch/include/ATen/ops/histc_ops.h 2024-08-08T20:48:49.6837987Z inflating: build/lib/torch/include/ATen/ops/histogram.h 2024-08-08T20:48:49.6839247Z inflating: build/lib/torch/include/ATen/ops/histogram_cpu_dispatch.h 2024-08-08T20:48:49.6840231Z inflating: build/lib/torch/include/ATen/ops/histogram_native.h 2024-08-08T20:48:49.6841460Z inflating: build/lib/torch/include/ATen/ops/histogram_ops.h 2024-08-08T20:48:49.6842468Z inflating: build/lib/torch/include/ATen/ops/histogramdd.h 2024-08-08T20:48:49.6844129Z inflating: build/lib/torch/include/ATen/ops/histogramdd_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6844727Z inflating: build/lib/torch/include/ATen/ops/histogramdd_native.h 2024-08-08T20:48:49.6845635Z inflating: build/lib/torch/include/ATen/ops/histogramdd_ops.h 2024-08-08T20:48:49.6846449Z inflating: build/lib/torch/include/ATen/ops/hsplit.h 2024-08-08T20:48:49.6847968Z inflating: build/lib/torch/include/ATen/ops/hsplit_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6848506Z inflating: build/lib/torch/include/ATen/ops/hsplit_native.h 2024-08-08T20:48:49.6849243Z inflating: build/lib/torch/include/ATen/ops/hsplit_ops.h 2024-08-08T20:48:49.6850419Z inflating: build/lib/torch/include/ATen/ops/hspmm.h 2024-08-08T20:48:49.6851441Z inflating: build/lib/torch/include/ATen/ops/hspmm_native.h 2024-08-08T20:48:49.6852349Z inflating: build/lib/torch/include/ATen/ops/hspmm_ops.h 2024-08-08T20:48:49.6853537Z inflating: build/lib/torch/include/ATen/ops/hstack.h 2024-08-08T20:48:49.6855167Z inflating: build/lib/torch/include/ATen/ops/hstack_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6855694Z inflating: build/lib/torch/include/ATen/ops/hstack_native.h 2024-08-08T20:48:49.6856660Z inflating: build/lib/torch/include/ATen/ops/hstack_ops.h 2024-08-08T20:48:49.6857597Z inflating: build/lib/torch/include/ATen/ops/huber_loss.h 2024-08-08T20:48:49.6859001Z inflating: build/lib/torch/include/ATen/ops/huber_loss_backward.h 2024-08-08T20:48:49.6860425Z inflating: build/lib/torch/include/ATen/ops/huber_loss_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6861028Z inflating: build/lib/torch/include/ATen/ops/huber_loss_backward_cpu_dispatch.h 2024-08-08T20:48:49.6861671Z inflating: build/lib/torch/include/ATen/ops/huber_loss_backward_cuda_dispatch.h 2024-08-08T20:48:49.6862753Z inflating: build/lib/torch/include/ATen/ops/huber_loss_backward_native.h 2024-08-08T20:48:49.6863993Z inflating: build/lib/torch/include/ATen/ops/huber_loss_backward_ops.h 2024-08-08T20:48:49.6865147Z inflating: build/lib/torch/include/ATen/ops/huber_loss_cpu_dispatch.h 2024-08-08T20:48:49.6866094Z inflating: build/lib/torch/include/ATen/ops/huber_loss_cuda_dispatch.h 2024-08-08T20:48:49.6866959Z inflating: build/lib/torch/include/ATen/ops/huber_loss_native.h 2024-08-08T20:48:49.6868063Z inflating: build/lib/torch/include/ATen/ops/huber_loss_ops.h 2024-08-08T20:48:49.6868841Z inflating: build/lib/torch/include/ATen/ops/hypot.h 2024-08-08T20:48:49.6870585Z inflating: build/lib/torch/include/ATen/ops/hypot_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6871234Z inflating: build/lib/torch/include/ATen/ops/hypot_cpu_dispatch.h 2024-08-08T20:48:49.6871985Z inflating: build/lib/torch/include/ATen/ops/hypot_cuda_dispatch.h 2024-08-08T20:48:49.6872968Z inflating: build/lib/torch/include/ATen/ops/hypot_meta.h 2024-08-08T20:48:49.6873980Z inflating: build/lib/torch/include/ATen/ops/hypot_meta_dispatch.h 2024-08-08T20:48:49.6874837Z inflating: build/lib/torch/include/ATen/ops/hypot_native.h 2024-08-08T20:48:49.6875865Z inflating: build/lib/torch/include/ATen/ops/hypot_ops.h 2024-08-08T20:48:49.6876727Z inflating: build/lib/torch/include/ATen/ops/i0.h 2024-08-08T20:48:49.6878288Z inflating: build/lib/torch/include/ATen/ops/i0_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6879050Z inflating: build/lib/torch/include/ATen/ops/i0_cpu_dispatch.h 2024-08-08T20:48:49.6879869Z inflating: build/lib/torch/include/ATen/ops/i0_cuda_dispatch.h 2024-08-08T20:48:49.6880722Z inflating: build/lib/torch/include/ATen/ops/i0_meta.h 2024-08-08T20:48:49.6881759Z inflating: build/lib/torch/include/ATen/ops/i0_meta_dispatch.h 2024-08-08T20:48:49.6882718Z inflating: build/lib/torch/include/ATen/ops/i0_native.h 2024-08-08T20:48:49.6883692Z inflating: build/lib/torch/include/ATen/ops/i0_ops.h 2024-08-08T20:48:49.6884739Z inflating: build/lib/torch/include/ATen/ops/igamma.h 2024-08-08T20:48:49.6886246Z inflating: build/lib/torch/include/ATen/ops/igamma_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6887777Z inflating: build/lib/torch/include/ATen/ops/igamma_cpu_dispatch.h 2024-08-08T20:48:49.6888817Z inflating: build/lib/torch/include/ATen/ops/igamma_cuda_dispatch.h 2024-08-08T20:48:49.6889603Z inflating: build/lib/torch/include/ATen/ops/igamma_meta.h 2024-08-08T20:48:49.6890728Z inflating: build/lib/torch/include/ATen/ops/igamma_meta_dispatch.h 2024-08-08T20:48:49.6891579Z inflating: build/lib/torch/include/ATen/ops/igamma_native.h 2024-08-08T20:48:49.6892624Z inflating: build/lib/torch/include/ATen/ops/igamma_ops.h 2024-08-08T20:48:49.6893572Z inflating: build/lib/torch/include/ATen/ops/igammac.h 2024-08-08T20:48:49.6895273Z inflating: build/lib/torch/include/ATen/ops/igammac_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6895994Z inflating: build/lib/torch/include/ATen/ops/igammac_cpu_dispatch.h 2024-08-08T20:48:49.6896564Z inflating: build/lib/torch/include/ATen/ops/igammac_cuda_dispatch.h 2024-08-08T20:48:49.6897561Z inflating: build/lib/torch/include/ATen/ops/igammac_meta.h 2024-08-08T20:48:49.6898631Z inflating: build/lib/torch/include/ATen/ops/igammac_meta_dispatch.h 2024-08-08T20:48:49.6899465Z inflating: build/lib/torch/include/ATen/ops/igammac_native.h 2024-08-08T20:48:49.6900580Z inflating: build/lib/torch/include/ATen/ops/igammac_ops.h 2024-08-08T20:48:49.6901470Z inflating: build/lib/torch/include/ATen/ops/im2col.h 2024-08-08T20:48:49.6902661Z inflating: build/lib/torch/include/ATen/ops/im2col_cpu_dispatch.h 2024-08-08T20:48:49.6903690Z inflating: build/lib/torch/include/ATen/ops/im2col_cuda_dispatch.h 2024-08-08T20:48:49.6904551Z inflating: build/lib/torch/include/ATen/ops/im2col_native.h 2024-08-08T20:48:49.6905715Z inflating: build/lib/torch/include/ATen/ops/im2col_ops.h 2024-08-08T20:48:49.6906607Z inflating: build/lib/torch/include/ATen/ops/imag.h 2024-08-08T20:48:49.6907994Z inflating: build/lib/torch/include/ATen/ops/imag_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6908715Z inflating: build/lib/torch/include/ATen/ops/imag_native.h 2024-08-08T20:48:49.6909539Z inflating: build/lib/torch/include/ATen/ops/imag_ops.h 2024-08-08T20:48:49.6910540Z inflating: build/lib/torch/include/ATen/ops/index.h 2024-08-08T20:48:49.6911545Z inflating: build/lib/torch/include/ATen/ops/index_add.h 2024-08-08T20:48:49.6913181Z inflating: build/lib/torch/include/ATen/ops/index_add_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6914231Z inflating: build/lib/torch/include/ATen/ops/index_add_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6914887Z inflating: build/lib/torch/include/ATen/ops/index_add_cpu_dispatch.h 2024-08-08T20:48:49.6915970Z inflating: build/lib/torch/include/ATen/ops/index_add_cuda_dispatch.h 2024-08-08T20:48:49.6916693Z inflating: build/lib/torch/include/ATen/ops/index_add_meta.h 2024-08-08T20:48:49.6917736Z inflating: build/lib/torch/include/ATen/ops/index_add_meta_dispatch.h 2024-08-08T20:48:49.6918840Z inflating: build/lib/torch/include/ATen/ops/index_add_native.h 2024-08-08T20:48:49.6920026Z inflating: build/lib/torch/include/ATen/ops/index_add_ops.h 2024-08-08T20:48:49.6921557Z inflating: build/lib/torch/include/ATen/ops/index_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6922007Z inflating: build/lib/torch/include/ATen/ops/index_copy.h 2024-08-08T20:48:49.6923366Z inflating: build/lib/torch/include/ATen/ops/index_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6924578Z inflating: build/lib/torch/include/ATen/ops/index_copy_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6925289Z inflating: build/lib/torch/include/ATen/ops/index_copy_cpu_dispatch.h 2024-08-08T20:48:49.6926095Z inflating: build/lib/torch/include/ATen/ops/index_copy_cuda_dispatch.h 2024-08-08T20:48:49.6927181Z inflating: build/lib/torch/include/ATen/ops/index_copy_meta.h 2024-08-08T20:48:49.6928143Z inflating: build/lib/torch/include/ATen/ops/index_copy_meta_dispatch.h 2024-08-08T20:48:49.6928978Z inflating: build/lib/torch/include/ATen/ops/index_copy_native.h 2024-08-08T20:48:49.6930213Z inflating: build/lib/torch/include/ATen/ops/index_copy_ops.h 2024-08-08T20:48:49.6931225Z inflating: build/lib/torch/include/ATen/ops/index_cpu_dispatch.h 2024-08-08T20:48:49.6932239Z inflating: build/lib/torch/include/ATen/ops/index_cuda_dispatch.h 2024-08-08T20:48:49.6933227Z inflating: build/lib/torch/include/ATen/ops/index_fill.h 2024-08-08T20:48:49.6934709Z inflating: build/lib/torch/include/ATen/ops/index_fill_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6935469Z inflating: build/lib/torch/include/ATen/ops/index_fill_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6936150Z inflating: build/lib/torch/include/ATen/ops/index_fill_cpu_dispatch.h 2024-08-08T20:48:49.6937308Z inflating: build/lib/torch/include/ATen/ops/index_fill_cuda_dispatch.h 2024-08-08T20:48:49.6938259Z inflating: build/lib/torch/include/ATen/ops/index_fill_meta_dispatch.h 2024-08-08T20:48:49.6939206Z inflating: build/lib/torch/include/ATen/ops/index_fill_native.h 2024-08-08T20:48:49.6940630Z inflating: build/lib/torch/include/ATen/ops/index_fill_ops.h 2024-08-08T20:48:49.6941618Z inflating: build/lib/torch/include/ATen/ops/index_meta.h 2024-08-08T20:48:49.6942714Z inflating: build/lib/torch/include/ATen/ops/index_meta_dispatch.h 2024-08-08T20:48:49.6943611Z inflating: build/lib/torch/include/ATen/ops/index_native.h 2024-08-08T20:48:49.6944634Z inflating: build/lib/torch/include/ATen/ops/index_ops.h 2024-08-08T20:48:49.6945672Z inflating: build/lib/torch/include/ATen/ops/index_put.h 2024-08-08T20:48:49.6947145Z inflating: build/lib/torch/include/ATen/ops/index_put_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6947932Z inflating: build/lib/torch/include/ATen/ops/index_put_native.h 2024-08-08T20:48:49.6948901Z inflating: build/lib/torch/include/ATen/ops/index_put_ops.h 2024-08-08T20:48:49.6949871Z inflating: build/lib/torch/include/ATen/ops/index_reduce.h 2024-08-08T20:48:49.6951364Z inflating: build/lib/torch/include/ATen/ops/index_reduce_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6952029Z inflating: build/lib/torch/include/ATen/ops/index_reduce_cpu_dispatch.h 2024-08-08T20:48:49.6953701Z inflating: build/lib/torch/include/ATen/ops/index_reduce_cuda_dispatch.h 2024-08-08T20:48:49.6954710Z inflating: build/lib/torch/include/ATen/ops/index_reduce_meta.h 2024-08-08T20:48:49.6955762Z inflating: build/lib/torch/include/ATen/ops/index_reduce_meta_dispatch.h 2024-08-08T20:48:49.6956733Z inflating: build/lib/torch/include/ATen/ops/index_reduce_native.h 2024-08-08T20:48:49.6957871Z inflating: build/lib/torch/include/ATen/ops/index_reduce_ops.h 2024-08-08T20:48:49.6959254Z inflating: build/lib/torch/include/ATen/ops/index_select.h 2024-08-08T20:48:49.6960320Z inflating: build/lib/torch/include/ATen/ops/index_select_backward.h 2024-08-08T20:48:49.6961737Z inflating: build/lib/torch/include/ATen/ops/index_select_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6962687Z inflating: build/lib/torch/include/ATen/ops/index_select_backward_native.h 2024-08-08T20:48:49.6963494Z inflating: build/lib/torch/include/ATen/ops/index_select_backward_ops.h 2024-08-08T20:48:49.6964850Z inflating: build/lib/torch/include/ATen/ops/index_select_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6965431Z inflating: build/lib/torch/include/ATen/ops/index_select_cpu_dispatch.h 2024-08-08T20:48:49.6966135Z inflating: build/lib/torch/include/ATen/ops/index_select_cuda_dispatch.h 2024-08-08T20:48:49.6967206Z inflating: build/lib/torch/include/ATen/ops/index_select_native.h 2024-08-08T20:48:49.6968386Z inflating: build/lib/torch/include/ATen/ops/index_select_ops.h 2024-08-08T20:48:49.6969074Z inflating: build/lib/torch/include/ATen/ops/indices.h 2024-08-08T20:48:49.6970427Z inflating: build/lib/torch/include/ATen/ops/indices_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6971205Z inflating: build/lib/torch/include/ATen/ops/indices_copy.h 2024-08-08T20:48:49.6972948Z inflating: build/lib/torch/include/ATen/ops/indices_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6973858Z inflating: build/lib/torch/include/ATen/ops/indices_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.6974232Z inflating: build/lib/torch/include/ATen/ops/indices_copy_native.h 2024-08-08T20:48:49.6975169Z inflating: build/lib/torch/include/ATen/ops/indices_copy_ops.h 2024-08-08T20:48:49.6976048Z inflating: build/lib/torch/include/ATen/ops/indices_native.h 2024-08-08T20:48:49.6977092Z inflating: build/lib/torch/include/ATen/ops/indices_ops.h 2024-08-08T20:48:49.6978207Z inflating: build/lib/torch/include/ATen/ops/infinitely_differentiable_gelu_backward.h 2024-08-08T20:48:49.6979549Z inflating: build/lib/torch/include/ATen/ops/infinitely_differentiable_gelu_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6980530Z inflating: build/lib/torch/include/ATen/ops/infinitely_differentiable_gelu_backward_native.h 2024-08-08T20:48:49.6981725Z inflating: build/lib/torch/include/ATen/ops/infinitely_differentiable_gelu_backward_ops.h 2024-08-08T20:48:49.6982415Z inflating: build/lib/torch/include/ATen/ops/inner.h 2024-08-08T20:48:49.6983689Z inflating: build/lib/torch/include/ATen/ops/inner_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6984394Z inflating: build/lib/torch/include/ATen/ops/inner_native.h 2024-08-08T20:48:49.6985449Z inflating: build/lib/torch/include/ATen/ops/inner_ops.h 2024-08-08T20:48:49.6986462Z inflating: build/lib/torch/include/ATen/ops/instance_norm.h 2024-08-08T20:48:49.6988151Z inflating: build/lib/torch/include/ATen/ops/instance_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6988753Z inflating: build/lib/torch/include/ATen/ops/instance_norm_native.h 2024-08-08T20:48:49.6989664Z inflating: build/lib/torch/include/ATen/ops/instance_norm_ops.h 2024-08-08T20:48:49.6990521Z inflating: build/lib/torch/include/ATen/ops/int_repr.h 2024-08-08T20:48:49.6991890Z inflating: build/lib/torch/include/ATen/ops/int_repr_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.6992634Z inflating: build/lib/torch/include/ATen/ops/int_repr_native.h 2024-08-08T20:48:49.6993633Z inflating: build/lib/torch/include/ATen/ops/int_repr_ops.h 2024-08-08T20:48:49.6994527Z inflating: build/lib/torch/include/ATen/ops/inverse.h 2024-08-08T20:48:49.6995989Z inflating: build/lib/torch/include/ATen/ops/inverse_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.6996833Z inflating: build/lib/torch/include/ATen/ops/inverse_native.h 2024-08-08T20:48:49.6997530Z inflating: build/lib/torch/include/ATen/ops/inverse_ops.h 2024-08-08T20:48:49.6998526Z inflating: build/lib/torch/include/ATen/ops/is_coalesced.h 2024-08-08T20:48:49.6999825Z inflating: build/lib/torch/include/ATen/ops/is_coalesced_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7000760Z inflating: build/lib/torch/include/ATen/ops/is_coalesced_native.h 2024-08-08T20:48:49.7001679Z inflating: build/lib/torch/include/ATen/ops/is_coalesced_ops.h 2024-08-08T20:48:49.7002618Z inflating: build/lib/torch/include/ATen/ops/is_complex.h 2024-08-08T20:48:49.7004216Z inflating: build/lib/torch/include/ATen/ops/is_complex_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7004767Z inflating: build/lib/torch/include/ATen/ops/is_complex_native.h 2024-08-08T20:48:49.7005362Z inflating: build/lib/torch/include/ATen/ops/is_complex_ops.h 2024-08-08T20:48:49.7006352Z inflating: build/lib/torch/include/ATen/ops/is_conj.h 2024-08-08T20:48:49.7008028Z inflating: build/lib/torch/include/ATen/ops/is_conj_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7008828Z inflating: build/lib/torch/include/ATen/ops/is_conj_native.h 2024-08-08T20:48:49.7009489Z inflating: build/lib/torch/include/ATen/ops/is_conj_ops.h 2024-08-08T20:48:49.7010548Z inflating: build/lib/torch/include/ATen/ops/is_distributed.h 2024-08-08T20:48:49.7012043Z inflating: build/lib/torch/include/ATen/ops/is_distributed_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7012750Z inflating: build/lib/torch/include/ATen/ops/is_distributed_native.h 2024-08-08T20:48:49.7013496Z inflating: build/lib/torch/include/ATen/ops/is_distributed_ops.h 2024-08-08T20:48:49.7014441Z inflating: build/lib/torch/include/ATen/ops/is_floating_point.h 2024-08-08T20:48:49.7015928Z inflating: build/lib/torch/include/ATen/ops/is_floating_point_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7016533Z inflating: build/lib/torch/include/ATen/ops/is_floating_point_native.h 2024-08-08T20:48:49.7017457Z inflating: build/lib/torch/include/ATen/ops/is_floating_point_ops.h 2024-08-08T20:48:49.7018909Z inflating: build/lib/torch/include/ATen/ops/is_inference.h 2024-08-08T20:48:49.7020540Z inflating: build/lib/torch/include/ATen/ops/is_inference_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7021169Z inflating: build/lib/torch/include/ATen/ops/is_inference_native.h 2024-08-08T20:48:49.7022075Z inflating: build/lib/torch/include/ATen/ops/is_inference_ops.h 2024-08-08T20:48:49.7022931Z inflating: build/lib/torch/include/ATen/ops/is_leaf.h 2024-08-08T20:48:49.7024352Z inflating: build/lib/torch/include/ATen/ops/is_leaf_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7024851Z inflating: build/lib/torch/include/ATen/ops/is_leaf_native.h 2024-08-08T20:48:49.7025865Z inflating: build/lib/torch/include/ATen/ops/is_leaf_ops.h 2024-08-08T20:48:49.7026767Z inflating: build/lib/torch/include/ATen/ops/is_neg.h 2024-08-08T20:48:49.7028204Z inflating: build/lib/torch/include/ATen/ops/is_neg_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7028660Z inflating: build/lib/torch/include/ATen/ops/is_neg_native.h 2024-08-08T20:48:49.7029681Z inflating: build/lib/torch/include/ATen/ops/is_neg_ops.h 2024-08-08T20:48:49.7030646Z inflating: build/lib/torch/include/ATen/ops/is_nonzero.h 2024-08-08T20:48:49.7032128Z inflating: build/lib/torch/include/ATen/ops/is_nonzero_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7032673Z inflating: build/lib/torch/include/ATen/ops/is_nonzero_native.h 2024-08-08T20:48:49.7033605Z inflating: build/lib/torch/include/ATen/ops/is_nonzero_ops.h 2024-08-08T20:48:49.7034534Z inflating: build/lib/torch/include/ATen/ops/is_pinned.h 2024-08-08T20:48:49.7035951Z inflating: build/lib/torch/include/ATen/ops/is_pinned_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7036738Z inflating: build/lib/torch/include/ATen/ops/is_pinned_native.h 2024-08-08T20:48:49.7037591Z inflating: build/lib/torch/include/ATen/ops/is_pinned_ops.h 2024-08-08T20:48:49.7038486Z inflating: build/lib/torch/include/ATen/ops/is_same_size.h 2024-08-08T20:48:49.7039997Z inflating: build/lib/torch/include/ATen/ops/is_same_size_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7040566Z inflating: build/lib/torch/include/ATen/ops/is_same_size_native.h 2024-08-08T20:48:49.7041468Z inflating: build/lib/torch/include/ATen/ops/is_same_size_ops.h 2024-08-08T20:48:49.7042407Z inflating: build/lib/torch/include/ATen/ops/is_set_to.h 2024-08-08T20:48:49.7043464Z inflating: build/lib/torch/include/ATen/ops/is_set_to_cpu_dispatch.h 2024-08-08T20:48:49.7044448Z inflating: build/lib/torch/include/ATen/ops/is_set_to_cuda_dispatch.h 2024-08-08T20:48:49.7045310Z inflating: build/lib/torch/include/ATen/ops/is_set_to_native.h 2024-08-08T20:48:49.7046313Z inflating: build/lib/torch/include/ATen/ops/is_set_to_ops.h 2024-08-08T20:48:49.7047196Z inflating: build/lib/torch/include/ATen/ops/is_signed.h 2024-08-08T20:48:49.7048633Z inflating: build/lib/torch/include/ATen/ops/is_signed_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7049392Z inflating: build/lib/torch/include/ATen/ops/is_signed_native.h 2024-08-08T20:48:49.7050338Z inflating: build/lib/torch/include/ATen/ops/is_signed_ops.h 2024-08-08T20:48:49.7051380Z inflating: build/lib/torch/include/ATen/ops/is_vulkan_available.h 2024-08-08T20:48:49.7052952Z inflating: build/lib/torch/include/ATen/ops/is_vulkan_available_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7053687Z inflating: build/lib/torch/include/ATen/ops/is_vulkan_available_native.h 2024-08-08T20:48:49.7054616Z inflating: build/lib/torch/include/ATen/ops/is_vulkan_available_ops.h 2024-08-08T20:48:49.7055308Z inflating: build/lib/torch/include/ATen/ops/isclose.h 2024-08-08T20:48:49.7056660Z inflating: build/lib/torch/include/ATen/ops/isclose_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7057146Z inflating: build/lib/torch/include/ATen/ops/isclose_native.h 2024-08-08T20:48:49.7057813Z inflating: build/lib/torch/include/ATen/ops/isclose_ops.h 2024-08-08T20:48:49.7058977Z inflating: build/lib/torch/include/ATen/ops/isfinite.h 2024-08-08T20:48:49.7060368Z inflating: build/lib/torch/include/ATen/ops/isfinite_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7060941Z inflating: build/lib/torch/include/ATen/ops/isfinite_native.h 2024-08-08T20:48:49.7061985Z inflating: build/lib/torch/include/ATen/ops/isfinite_ops.h 2024-08-08T20:48:49.7063078Z inflating: build/lib/torch/include/ATen/ops/isin.h 2024-08-08T20:48:49.7064805Z inflating: build/lib/torch/include/ATen/ops/isin_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7065576Z inflating: build/lib/torch/include/ATen/ops/isin_cpu_dispatch.h 2024-08-08T20:48:49.7066622Z inflating: build/lib/torch/include/ATen/ops/isin_cuda_dispatch.h 2024-08-08T20:48:49.7067443Z inflating: build/lib/torch/include/ATen/ops/isin_meta.h 2024-08-08T20:48:49.7068654Z inflating: build/lib/torch/include/ATen/ops/isin_meta_dispatch.h 2024-08-08T20:48:49.7069562Z inflating: build/lib/torch/include/ATen/ops/isin_native.h 2024-08-08T20:48:49.7070771Z inflating: build/lib/torch/include/ATen/ops/isin_ops.h 2024-08-08T20:48:49.7071694Z inflating: build/lib/torch/include/ATen/ops/isinf.h 2024-08-08T20:48:49.7073081Z inflating: build/lib/torch/include/ATen/ops/isinf_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7073788Z inflating: build/lib/torch/include/ATen/ops/isinf_native.h 2024-08-08T20:48:49.7074798Z inflating: build/lib/torch/include/ATen/ops/isinf_ops.h 2024-08-08T20:48:49.7075651Z inflating: build/lib/torch/include/ATen/ops/isnan.h 2024-08-08T20:48:49.7077145Z inflating: build/lib/torch/include/ATen/ops/isnan_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7078097Z inflating: build/lib/torch/include/ATen/ops/isnan_cpu_dispatch.h 2024-08-08T20:48:49.7078569Z inflating: build/lib/torch/include/ATen/ops/isnan_cuda_dispatch.h 2024-08-08T20:48:49.7079512Z inflating: build/lib/torch/include/ATen/ops/isnan_native.h 2024-08-08T20:48:49.7080652Z inflating: build/lib/torch/include/ATen/ops/isnan_ops.h 2024-08-08T20:48:49.7081748Z inflating: build/lib/torch/include/ATen/ops/isneginf.h 2024-08-08T20:48:49.7084215Z inflating: build/lib/torch/include/ATen/ops/isneginf_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7084910Z inflating: build/lib/torch/include/ATen/ops/isneginf_cpu_dispatch.h 2024-08-08T20:48:49.7085505Z inflating: build/lib/torch/include/ATen/ops/isneginf_cuda_dispatch.h 2024-08-08T20:48:49.7086225Z inflating: build/lib/torch/include/ATen/ops/isneginf_meta.h 2024-08-08T20:48:49.7087591Z inflating: build/lib/torch/include/ATen/ops/isneginf_meta_dispatch.h 2024-08-08T20:48:49.7088458Z inflating: build/lib/torch/include/ATen/ops/isneginf_native.h 2024-08-08T20:48:49.7089534Z inflating: build/lib/torch/include/ATen/ops/isneginf_ops.h 2024-08-08T20:48:49.7090581Z inflating: build/lib/torch/include/ATen/ops/isposinf.h 2024-08-08T20:48:49.7092115Z inflating: build/lib/torch/include/ATen/ops/isposinf_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7092840Z inflating: build/lib/torch/include/ATen/ops/isposinf_cpu_dispatch.h 2024-08-08T20:48:49.7093738Z inflating: build/lib/torch/include/ATen/ops/isposinf_cuda_dispatch.h 2024-08-08T20:48:49.7094466Z inflating: build/lib/torch/include/ATen/ops/isposinf_meta.h 2024-08-08T20:48:49.7095512Z inflating: build/lib/torch/include/ATen/ops/isposinf_meta_dispatch.h 2024-08-08T20:48:49.7096394Z inflating: build/lib/torch/include/ATen/ops/isposinf_native.h 2024-08-08T20:48:49.7097408Z inflating: build/lib/torch/include/ATen/ops/isposinf_ops.h 2024-08-08T20:48:49.7098318Z inflating: build/lib/torch/include/ATen/ops/isreal.h 2024-08-08T20:48:49.7099776Z inflating: build/lib/torch/include/ATen/ops/isreal_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7100279Z inflating: build/lib/torch/include/ATen/ops/isreal_native.h 2024-08-08T20:48:49.7101076Z inflating: build/lib/torch/include/ATen/ops/isreal_ops.h 2024-08-08T20:48:49.7102246Z inflating: build/lib/torch/include/ATen/ops/istft.h 2024-08-08T20:48:49.7103688Z inflating: build/lib/torch/include/ATen/ops/istft_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7104401Z inflating: build/lib/torch/include/ATen/ops/istft_native.h 2024-08-08T20:48:49.7105427Z inflating: build/lib/torch/include/ATen/ops/istft_ops.h 2024-08-08T20:48:49.7106279Z inflating: build/lib/torch/include/ATen/ops/item.h 2024-08-08T20:48:49.7107652Z inflating: build/lib/torch/include/ATen/ops/item_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7108390Z inflating: build/lib/torch/include/ATen/ops/item_native.h 2024-08-08T20:48:49.7109249Z inflating: build/lib/torch/include/ATen/ops/item_ops.h 2024-08-08T20:48:49.7110534Z inflating: build/lib/torch/include/ATen/ops/kaiser_window.h 2024-08-08T20:48:49.7112079Z inflating: build/lib/torch/include/ATen/ops/kaiser_window_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7112599Z inflating: build/lib/torch/include/ATen/ops/kaiser_window_native.h 2024-08-08T20:48:49.7113652Z inflating: build/lib/torch/include/ATen/ops/kaiser_window_ops.h 2024-08-08T20:48:49.7114719Z inflating: build/lib/torch/include/ATen/ops/kl_div.h 2024-08-08T20:48:49.7115932Z inflating: build/lib/torch/include/ATen/ops/kl_div_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7116788Z inflating: build/lib/torch/include/ATen/ops/kl_div_native.h 2024-08-08T20:48:49.7117697Z inflating: build/lib/torch/include/ATen/ops/kl_div_ops.h 2024-08-08T20:48:49.7118715Z inflating: build/lib/torch/include/ATen/ops/kron.h 2024-08-08T20:48:49.7120139Z inflating: build/lib/torch/include/ATen/ops/kron_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7120741Z inflating: build/lib/torch/include/ATen/ops/kron_native.h 2024-08-08T20:48:49.7121844Z inflating: build/lib/torch/include/ATen/ops/kron_ops.h 2024-08-08T20:48:49.7122978Z inflating: build/lib/torch/include/ATen/ops/kthvalue.h 2024-08-08T20:48:49.7124341Z inflating: build/lib/torch/include/ATen/ops/kthvalue_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7125246Z inflating: build/lib/torch/include/ATen/ops/kthvalue_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7126161Z inflating: build/lib/torch/include/ATen/ops/kthvalue_cpu_dispatch.h 2024-08-08T20:48:49.7127362Z inflating: build/lib/torch/include/ATen/ops/kthvalue_cuda_dispatch.h 2024-08-08T20:48:49.7128439Z inflating: build/lib/torch/include/ATen/ops/kthvalue_native.h 2024-08-08T20:48:49.7129596Z inflating: build/lib/torch/include/ATen/ops/kthvalue_ops.h 2024-08-08T20:48:49.7130526Z inflating: build/lib/torch/include/ATen/ops/l1_loss.h 2024-08-08T20:48:49.7131922Z inflating: build/lib/torch/include/ATen/ops/l1_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7133292Z inflating: build/lib/torch/include/ATen/ops/l1_loss_native.h 2024-08-08T20:48:49.7133826Z inflating: build/lib/torch/include/ATen/ops/l1_loss_ops.h 2024-08-08T20:48:49.7134587Z inflating: build/lib/torch/include/ATen/ops/layer_norm.h 2024-08-08T20:48:49.7136050Z inflating: build/lib/torch/include/ATen/ops/layer_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7136820Z inflating: build/lib/torch/include/ATen/ops/layer_norm_native.h 2024-08-08T20:48:49.7137749Z inflating: build/lib/torch/include/ATen/ops/layer_norm_ops.h 2024-08-08T20:48:49.7138605Z inflating: build/lib/torch/include/ATen/ops/lcm.h 2024-08-08T20:48:49.7140207Z inflating: build/lib/torch/include/ATen/ops/lcm_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7140669Z inflating: build/lib/torch/include/ATen/ops/lcm_cpu_dispatch.h 2024-08-08T20:48:49.7141368Z inflating: build/lib/torch/include/ATen/ops/lcm_cuda_dispatch.h 2024-08-08T20:48:49.7142346Z inflating: build/lib/torch/include/ATen/ops/lcm_meta.h 2024-08-08T20:48:49.7143345Z inflating: build/lib/torch/include/ATen/ops/lcm_meta_dispatch.h 2024-08-08T20:48:49.7144357Z inflating: build/lib/torch/include/ATen/ops/lcm_native.h 2024-08-08T20:48:49.7145509Z inflating: build/lib/torch/include/ATen/ops/lcm_ops.h 2024-08-08T20:48:49.7146562Z inflating: build/lib/torch/include/ATen/ops/ldexp.h 2024-08-08T20:48:49.7148103Z inflating: build/lib/torch/include/ATen/ops/ldexp_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7149325Z inflating: build/lib/torch/include/ATen/ops/ldexp_native.h 2024-08-08T20:48:49.7150492Z inflating: build/lib/torch/include/ATen/ops/ldexp_ops.h 2024-08-08T20:48:49.7151424Z inflating: build/lib/torch/include/ATen/ops/le.h 2024-08-08T20:48:49.7153107Z inflating: build/lib/torch/include/ATen/ops/le_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7153935Z inflating: build/lib/torch/include/ATen/ops/le_cpu_dispatch.h 2024-08-08T20:48:49.7154737Z inflating: build/lib/torch/include/ATen/ops/le_cuda_dispatch.h 2024-08-08T20:48:49.7155599Z inflating: build/lib/torch/include/ATen/ops/le_meta.h 2024-08-08T20:48:49.7156792Z inflating: build/lib/torch/include/ATen/ops/le_meta_dispatch.h 2024-08-08T20:48:49.7157724Z inflating: build/lib/torch/include/ATen/ops/le_native.h 2024-08-08T20:48:49.7159085Z inflating: build/lib/torch/include/ATen/ops/le_ops.h 2024-08-08T20:48:49.7161615Z inflating: build/lib/torch/include/ATen/ops/leaky_relu.h 2024-08-08T20:48:49.7162307Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_backward.h 2024-08-08T20:48:49.7163769Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7164350Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_backward_cpu_dispatch.h 2024-08-08T20:48:49.7165527Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_backward_cuda_dispatch.h 2024-08-08T20:48:49.7166350Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_backward_meta.h 2024-08-08T20:48:49.7167607Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_backward_meta_dispatch.h 2024-08-08T20:48:49.7168722Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_backward_native.h 2024-08-08T20:48:49.7170062Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_backward_ops.h 2024-08-08T20:48:49.7171570Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7172308Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_cpu_dispatch.h 2024-08-08T20:48:49.7173222Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_cuda_dispatch.h 2024-08-08T20:48:49.7173933Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_meta.h 2024-08-08T20:48:49.7175153Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_meta_dispatch.h 2024-08-08T20:48:49.7176025Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_native.h 2024-08-08T20:48:49.7177297Z inflating: build/lib/torch/include/ATen/ops/leaky_relu_ops.h 2024-08-08T20:48:49.7178086Z inflating: build/lib/torch/include/ATen/ops/lerp.h 2024-08-08T20:48:49.7179663Z inflating: build/lib/torch/include/ATen/ops/lerp_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7180159Z inflating: build/lib/torch/include/ATen/ops/lerp_cpu_dispatch.h 2024-08-08T20:48:49.7181077Z inflating: build/lib/torch/include/ATen/ops/lerp_cuda_dispatch.h 2024-08-08T20:48:49.7182032Z inflating: build/lib/torch/include/ATen/ops/lerp_meta.h 2024-08-08T20:48:49.7183431Z inflating: build/lib/torch/include/ATen/ops/lerp_meta_dispatch.h 2024-08-08T20:48:49.7184339Z inflating: build/lib/torch/include/ATen/ops/lerp_native.h 2024-08-08T20:48:49.7185576Z inflating: build/lib/torch/include/ATen/ops/lerp_ops.h 2024-08-08T20:48:49.7186518Z inflating: build/lib/torch/include/ATen/ops/less.h 2024-08-08T20:48:49.7187957Z inflating: build/lib/torch/include/ATen/ops/less_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7188621Z inflating: build/lib/torch/include/ATen/ops/less_equal.h 2024-08-08T20:48:49.7190281Z inflating: build/lib/torch/include/ATen/ops/less_equal_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7190810Z inflating: build/lib/torch/include/ATen/ops/less_equal_native.h 2024-08-08T20:48:49.7191715Z inflating: build/lib/torch/include/ATen/ops/less_equal_ops.h 2024-08-08T20:48:49.7192527Z inflating: build/lib/torch/include/ATen/ops/less_native.h 2024-08-08T20:48:49.7194027Z inflating: build/lib/torch/include/ATen/ops/less_ops.h 2024-08-08T20:48:49.7194803Z inflating: build/lib/torch/include/ATen/ops/lgamma.h 2024-08-08T20:48:49.7196680Z inflating: build/lib/torch/include/ATen/ops/lgamma_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7197315Z inflating: build/lib/torch/include/ATen/ops/lgamma_cpu_dispatch.h 2024-08-08T20:48:49.7198202Z inflating: build/lib/torch/include/ATen/ops/lgamma_cuda_dispatch.h 2024-08-08T20:48:49.7199033Z inflating: build/lib/torch/include/ATen/ops/lgamma_meta.h 2024-08-08T20:48:49.7200179Z inflating: build/lib/torch/include/ATen/ops/lgamma_meta_dispatch.h 2024-08-08T20:48:49.7201070Z inflating: build/lib/torch/include/ATen/ops/lgamma_native.h 2024-08-08T20:48:49.7202289Z inflating: build/lib/torch/include/ATen/ops/lgamma_ops.h 2024-08-08T20:48:49.7203195Z inflating: build/lib/torch/include/ATen/ops/lift.h 2024-08-08T20:48:49.7204626Z inflating: build/lib/torch/include/ATen/ops/lift_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7205418Z inflating: build/lib/torch/include/ATen/ops/lift_fresh.h 2024-08-08T20:48:49.7206788Z inflating: build/lib/torch/include/ATen/ops/lift_fresh_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7207597Z inflating: build/lib/torch/include/ATen/ops/lift_fresh_copy.h 2024-08-08T20:48:49.7208786Z inflating: build/lib/torch/include/ATen/ops/lift_fresh_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7210180Z inflating: build/lib/torch/include/ATen/ops/lift_fresh_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7210853Z inflating: build/lib/torch/include/ATen/ops/lift_fresh_copy_native.h 2024-08-08T20:48:49.7211645Z inflating: build/lib/torch/include/ATen/ops/lift_fresh_copy_ops.h 2024-08-08T20:48:49.7212470Z inflating: build/lib/torch/include/ATen/ops/lift_fresh_native.h 2024-08-08T20:48:49.7213561Z inflating: build/lib/torch/include/ATen/ops/lift_fresh_ops.h 2024-08-08T20:48:49.7214386Z inflating: build/lib/torch/include/ATen/ops/lift_native.h 2024-08-08T20:48:49.7215327Z inflating: build/lib/torch/include/ATen/ops/lift_ops.h 2024-08-08T20:48:49.7216648Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky.h 2024-08-08T20:48:49.7218977Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7219737Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_ex.h 2024-08-08T20:48:49.7221216Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_ex_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7222091Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_ex_cpu_dispatch.h 2024-08-08T20:48:49.7222866Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_ex_cuda_dispatch.h 2024-08-08T20:48:49.7223678Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_ex_meta.h 2024-08-08T20:48:49.7224789Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_ex_meta_dispatch.h 2024-08-08T20:48:49.7225640Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_ex_native.h 2024-08-08T20:48:49.7226737Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_ex_ops.h 2024-08-08T20:48:49.7227652Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_native.h 2024-08-08T20:48:49.7228739Z inflating: build/lib/torch/include/ATen/ops/linalg_cholesky_ops.h 2024-08-08T20:48:49.7229731Z inflating: build/lib/torch/include/ATen/ops/linalg_cond.h 2024-08-08T20:48:49.7231246Z inflating: build/lib/torch/include/ATen/ops/linalg_cond_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7232064Z inflating: build/lib/torch/include/ATen/ops/linalg_cond_native.h 2024-08-08T20:48:49.7233065Z inflating: build/lib/torch/include/ATen/ops/linalg_cond_ops.h 2024-08-08T20:48:49.7234028Z inflating: build/lib/torch/include/ATen/ops/linalg_cross.h 2024-08-08T20:48:49.7234824Z inflating: build/lib/torch/include/ATen/ops/mT_ops.h 2024-08-08T20:48:49.7236980Z inflating: build/lib/torch/include/ATen/ops/linalg_cross_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7237752Z inflating: build/lib/torch/include/ATen/ops/linalg_cross_cpu_dispatch.h 2024-08-08T20:48:49.7238700Z inflating: build/lib/torch/include/ATen/ops/linalg_cross_cuda_dispatch.h 2024-08-08T20:48:49.7239471Z inflating: build/lib/torch/include/ATen/ops/linalg_cross_meta.h 2024-08-08T20:48:49.7240427Z inflating: build/lib/torch/include/ATen/ops/linalg_cross_meta_dispatch.h 2024-08-08T20:48:49.7241307Z inflating: build/lib/torch/include/ATen/ops/linalg_cross_native.h 2024-08-08T20:48:49.7242376Z inflating: build/lib/torch/include/ATen/ops/linalg_cross_ops.h 2024-08-08T20:48:49.7243262Z inflating: build/lib/torch/include/ATen/ops/linalg_det.h 2024-08-08T20:48:49.7244703Z inflating: build/lib/torch/include/ATen/ops/linalg_det_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7245483Z inflating: build/lib/torch/include/ATen/ops/linalg_det_native.h 2024-08-08T20:48:49.7246347Z inflating: build/lib/torch/include/ATen/ops/linalg_det_ops.h 2024-08-08T20:48:49.7247347Z inflating: build/lib/torch/include/ATen/ops/linalg_diagonal.h 2024-08-08T20:48:49.7248856Z inflating: build/lib/torch/include/ATen/ops/linalg_diagonal_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7249444Z inflating: build/lib/torch/include/ATen/ops/linalg_diagonal_native.h 2024-08-08T20:48:49.7250502Z inflating: build/lib/torch/include/ATen/ops/linalg_diagonal_ops.h 2024-08-08T20:48:49.7251492Z inflating: build/lib/torch/include/ATen/ops/linalg_eig.h 2024-08-08T20:48:49.7252626Z inflating: build/lib/torch/include/ATen/ops/linalg_eig_cpu_dispatch.h 2024-08-08T20:48:49.7253660Z inflating: build/lib/torch/include/ATen/ops/linalg_eig_cuda_dispatch.h 2024-08-08T20:48:49.7254516Z inflating: build/lib/torch/include/ATen/ops/linalg_eig_native.h 2024-08-08T20:48:49.7255564Z inflating: build/lib/torch/include/ATen/ops/linalg_eig_ops.h 2024-08-08T20:48:49.7256570Z inflating: build/lib/torch/include/ATen/ops/linalg_eigh.h 2024-08-08T20:48:49.7258090Z inflating: build/lib/torch/include/ATen/ops/linalg_eigh_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7258944Z inflating: build/lib/torch/include/ATen/ops/linalg_eigh_native.h 2024-08-08T20:48:49.7259911Z inflating: build/lib/torch/include/ATen/ops/linalg_eigh_ops.h 2024-08-08T20:48:49.7261069Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvals.h 2024-08-08T20:48:49.7262506Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvals_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7263328Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvals_cpu_dispatch.h 2024-08-08T20:48:49.7264004Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvals_cuda_dispatch.h 2024-08-08T20:48:49.7264905Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvals_native.h 2024-08-08T20:48:49.7265978Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvals_ops.h 2024-08-08T20:48:49.7266936Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvalsh.h 2024-08-08T20:48:49.7268509Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvalsh_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7269194Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvalsh_native.h 2024-08-08T20:48:49.7270150Z inflating: build/lib/torch/include/ATen/ops/linalg_eigvalsh_ops.h 2024-08-08T20:48:49.7271037Z inflating: build/lib/torch/include/ATen/ops/linalg_householder_product.h 2024-08-08T20:48:49.7272229Z inflating: build/lib/torch/include/ATen/ops/linalg_householder_product_cpu_dispatch.h 2024-08-08T20:48:49.7273358Z inflating: build/lib/torch/include/ATen/ops/linalg_householder_product_cuda_dispatch.h 2024-08-08T20:48:49.7274173Z inflating: build/lib/torch/include/ATen/ops/linalg_householder_product_native.h 2024-08-08T20:48:49.7275131Z inflating: build/lib/torch/include/ATen/ops/linalg_householder_product_ops.h 2024-08-08T20:48:49.7275840Z inflating: build/lib/torch/include/ATen/ops/linalg_inv.h 2024-08-08T20:48:49.7277272Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7278061Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_ex.h 2024-08-08T20:48:49.7279560Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_ex_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7280342Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_ex_cpu_dispatch.h 2024-08-08T20:48:49.7281301Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_ex_cuda_dispatch.h 2024-08-08T20:48:49.7281949Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_ex_meta.h 2024-08-08T20:48:49.7283435Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_ex_meta_dispatch.h 2024-08-08T20:48:49.7285044Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_ex_native.h 2024-08-08T20:48:49.7286124Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_ex_ops.h 2024-08-08T20:48:49.7286977Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_native.h 2024-08-08T20:48:49.7288141Z inflating: build/lib/torch/include/ATen/ops/linalg_inv_ops.h 2024-08-08T20:48:49.7289244Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor.h 2024-08-08T20:48:49.7290668Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7291248Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_ex.h 2024-08-08T20:48:49.7292666Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_ex_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7293680Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_ex_cpu_dispatch.h 2024-08-08T20:48:49.7294773Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_ex_cuda_dispatch.h 2024-08-08T20:48:49.7295312Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_ex_meta.h 2024-08-08T20:48:49.7296075Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_ex_meta_dispatch.h 2024-08-08T20:48:49.7297123Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_ex_native.h 2024-08-08T20:48:49.7298228Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_ex_ops.h 2024-08-08T20:48:49.7299451Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_native.h 2024-08-08T20:48:49.7300565Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_factor_ops.h 2024-08-08T20:48:49.7301435Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_solve.h 2024-08-08T20:48:49.7303167Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_solve_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7303997Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_solve_cpu_dispatch.h 2024-08-08T20:48:49.7304563Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_solve_cuda_dispatch.h 2024-08-08T20:48:49.7305285Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_solve_meta.h 2024-08-08T20:48:49.7306624Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_solve_meta_dispatch.h 2024-08-08T20:48:49.7307445Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_solve_native.h 2024-08-08T20:48:49.7308518Z inflating: build/lib/torch/include/ATen/ops/linalg_ldl_solve_ops.h 2024-08-08T20:48:49.7309552Z inflating: build/lib/torch/include/ATen/ops/linalg_lstsq.h 2024-08-08T20:48:49.7310941Z inflating: build/lib/torch/include/ATen/ops/linalg_lstsq_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7311864Z inflating: build/lib/torch/include/ATen/ops/linalg_lstsq_cpu_dispatch.h 2024-08-08T20:48:49.7312671Z inflating: build/lib/torch/include/ATen/ops/linalg_lstsq_cuda_dispatch.h 2024-08-08T20:48:49.7313563Z inflating: build/lib/torch/include/ATen/ops/linalg_lstsq_native.h 2024-08-08T20:48:49.7314696Z inflating: build/lib/torch/include/ATen/ops/linalg_lstsq_ops.h 2024-08-08T20:48:49.7315498Z inflating: build/lib/torch/include/ATen/ops/linalg_lu.h 2024-08-08T20:48:49.7317448Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7318144Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_cpu_dispatch.h 2024-08-08T20:48:49.7318944Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_cuda_dispatch.h 2024-08-08T20:48:49.7319892Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor.h 2024-08-08T20:48:49.7321301Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7322158Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_ex.h 2024-08-08T20:48:49.7323490Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_ex_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7324308Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_ex_cpu_dispatch.h 2024-08-08T20:48:49.7325353Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_ex_cuda_dispatch.h 2024-08-08T20:48:49.7325961Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_ex_meta.h 2024-08-08T20:48:49.7326933Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_ex_meta_dispatch.h 2024-08-08T20:48:49.7328041Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_ex_native.h 2024-08-08T20:48:49.7329041Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_ex_ops.h 2024-08-08T20:48:49.7329960Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_native.h 2024-08-08T20:48:49.7331099Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_factor_ops.h 2024-08-08T20:48:49.7331892Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_meta.h 2024-08-08T20:48:49.7333022Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_meta_dispatch.h 2024-08-08T20:48:49.7333961Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_native.h 2024-08-08T20:48:49.7335056Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_ops.h 2024-08-08T20:48:49.7336046Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_solve.h 2024-08-08T20:48:49.7337669Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_solve_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7338685Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_solve_cpu_dispatch.h 2024-08-08T20:48:49.7339470Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_solve_cuda_dispatch.h 2024-08-08T20:48:49.7340323Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_solve_meta.h 2024-08-08T20:48:49.7341170Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_solve_meta_dispatch.h 2024-08-08T20:48:49.7342076Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_solve_native.h 2024-08-08T20:48:49.7343151Z inflating: build/lib/torch/include/ATen/ops/linalg_lu_solve_ops.h 2024-08-08T20:48:49.7344083Z inflating: build/lib/torch/include/ATen/ops/linalg_matmul.h 2024-08-08T20:48:49.7345602Z inflating: build/lib/torch/include/ATen/ops/linalg_matmul_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7346204Z inflating: build/lib/torch/include/ATen/ops/linalg_matmul_native.h 2024-08-08T20:48:49.7347139Z inflating: build/lib/torch/include/ATen/ops/linalg_matmul_ops.h 2024-08-08T20:48:49.7348044Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_exp.h 2024-08-08T20:48:49.7349468Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_exp_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7350929Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_exp_cpu_dispatch.h 2024-08-08T20:48:49.7352024Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_exp_cuda_dispatch.h 2024-08-08T20:48:49.7352779Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_exp_native.h 2024-08-08T20:48:49.7353976Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_exp_ops.h 2024-08-08T20:48:49.7355039Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_norm.h 2024-08-08T20:48:49.7356547Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7357120Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_norm_native.h 2024-08-08T20:48:49.7358102Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_norm_ops.h 2024-08-08T20:48:49.7359187Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_power.h 2024-08-08T20:48:49.7360605Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_power_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7361195Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_power_native.h 2024-08-08T20:48:49.7362287Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_power_ops.h 2024-08-08T20:48:49.7363452Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_rank.h 2024-08-08T20:48:49.7364827Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_rank_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7365786Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_rank_native.h 2024-08-08T20:48:49.7367438Z inflating: build/lib/torch/include/ATen/ops/linalg_matrix_rank_ops.h 2024-08-08T20:48:49.7367954Z inflating: build/lib/torch/include/ATen/ops/linalg_multi_dot.h 2024-08-08T20:48:49.7369198Z inflating: build/lib/torch/include/ATen/ops/linalg_multi_dot_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7370039Z inflating: build/lib/torch/include/ATen/ops/linalg_multi_dot_native.h 2024-08-08T20:48:49.7371212Z inflating: build/lib/torch/include/ATen/ops/linalg_multi_dot_ops.h 2024-08-08T20:48:49.7372234Z inflating: build/lib/torch/include/ATen/ops/linalg_norm.h 2024-08-08T20:48:49.7373722Z inflating: build/lib/torch/include/ATen/ops/linalg_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7374275Z inflating: build/lib/torch/include/ATen/ops/linalg_norm_native.h 2024-08-08T20:48:49.7375202Z inflating: build/lib/torch/include/ATen/ops/linalg_norm_ops.h 2024-08-08T20:48:49.7376608Z inflating: build/lib/torch/include/ATen/ops/linalg_pinv.h 2024-08-08T20:48:49.7377591Z inflating: build/lib/torch/include/ATen/ops/linalg_pinv_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7378971Z inflating: build/lib/torch/include/ATen/ops/linalg_pinv_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7379790Z inflating: build/lib/torch/include/ATen/ops/linalg_pinv_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7380453Z inflating: build/lib/torch/include/ATen/ops/linalg_pinv_native.h 2024-08-08T20:48:49.7382034Z inflating: build/lib/torch/include/ATen/ops/linalg_pinv_ops.h 2024-08-08T20:48:49.7382728Z inflating: build/lib/torch/include/ATen/ops/linalg_qr.h 2024-08-08T20:48:49.7384929Z inflating: build/lib/torch/include/ATen/ops/linalg_qr_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7385450Z inflating: build/lib/torch/include/ATen/ops/linalg_qr_cpu_dispatch.h 2024-08-08T20:48:49.7386519Z inflating: build/lib/torch/include/ATen/ops/linalg_qr_cuda_dispatch.h 2024-08-08T20:48:49.7387334Z inflating: build/lib/torch/include/ATen/ops/linalg_qr_meta.h 2024-08-08T20:48:49.7388452Z inflating: build/lib/torch/include/ATen/ops/linalg_qr_meta_dispatch.h 2024-08-08T20:48:49.7389548Z inflating: build/lib/torch/include/ATen/ops/linalg_qr_native.h 2024-08-08T20:48:49.7390700Z inflating: build/lib/torch/include/ATen/ops/linalg_qr_ops.h 2024-08-08T20:48:49.7391704Z inflating: build/lib/torch/include/ATen/ops/linalg_slogdet.h 2024-08-08T20:48:49.7393253Z inflating: build/lib/torch/include/ATen/ops/linalg_slogdet_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7393848Z inflating: build/lib/torch/include/ATen/ops/linalg_slogdet_native.h 2024-08-08T20:48:49.7394758Z inflating: build/lib/torch/include/ATen/ops/linalg_slogdet_ops.h 2024-08-08T20:48:49.7395682Z inflating: build/lib/torch/include/ATen/ops/linalg_solve.h 2024-08-08T20:48:49.7397069Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7397832Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_ex.h 2024-08-08T20:48:49.7399282Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_ex_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7399957Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_ex_native.h 2024-08-08T20:48:49.7400830Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_ex_ops.h 2024-08-08T20:48:49.7401730Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_native.h 2024-08-08T20:48:49.7402771Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_ops.h 2024-08-08T20:48:49.7403824Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_triangular.h 2024-08-08T20:48:49.7404966Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_triangular_cpu_dispatch.h 2024-08-08T20:48:49.7405996Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_triangular_cuda_dispatch.h 2024-08-08T20:48:49.7406997Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_triangular_native.h 2024-08-08T20:48:49.7408134Z inflating: build/lib/torch/include/ATen/ops/linalg_solve_triangular_ops.h 2024-08-08T20:48:49.7408926Z inflating: build/lib/torch/include/ATen/ops/linalg_svd.h 2024-08-08T20:48:49.7410433Z inflating: build/lib/torch/include/ATen/ops/linalg_svd_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7410957Z inflating: build/lib/torch/include/ATen/ops/linalg_svd_native.h 2024-08-08T20:48:49.7411853Z inflating: build/lib/torch/include/ATen/ops/linalg_svd_ops.h 2024-08-08T20:48:49.7413059Z inflating: build/lib/torch/include/ATen/ops/linalg_svdvals.h 2024-08-08T20:48:49.7414597Z inflating: build/lib/torch/include/ATen/ops/linalg_svdvals_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7415222Z inflating: build/lib/torch/include/ATen/ops/linalg_svdvals_native.h 2024-08-08T20:48:49.7415988Z inflating: build/lib/torch/include/ATen/ops/linalg_svdvals_ops.h 2024-08-08T20:48:49.7417083Z inflating: build/lib/torch/include/ATen/ops/linalg_tensorinv.h 2024-08-08T20:48:49.7419542Z inflating: build/lib/torch/include/ATen/ops/linalg_tensorinv_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7420383Z inflating: build/lib/torch/include/ATen/ops/linalg_tensorinv_native.h 2024-08-08T20:48:49.7421035Z inflating: build/lib/torch/include/ATen/ops/linalg_tensorinv_ops.h 2024-08-08T20:48:49.7421988Z inflating: build/lib/torch/include/ATen/ops/linalg_tensorsolve.h 2024-08-08T20:48:49.7423535Z inflating: build/lib/torch/include/ATen/ops/linalg_tensorsolve_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7424235Z inflating: build/lib/torch/include/ATen/ops/linalg_tensorsolve_native.h 2024-08-08T20:48:49.7425032Z inflating: build/lib/torch/include/ATen/ops/linalg_tensorsolve_ops.h 2024-08-08T20:48:49.7426015Z inflating: build/lib/torch/include/ATen/ops/linalg_vander.h 2024-08-08T20:48:49.7427548Z inflating: build/lib/torch/include/ATen/ops/linalg_vander_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7428193Z inflating: build/lib/torch/include/ATen/ops/linalg_vander_native.h 2024-08-08T20:48:49.7429037Z inflating: build/lib/torch/include/ATen/ops/linalg_vander_ops.h 2024-08-08T20:48:49.7430014Z inflating: build/lib/torch/include/ATen/ops/linalg_vecdot.h 2024-08-08T20:48:49.7431542Z inflating: build/lib/torch/include/ATen/ops/linalg_vecdot_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7432177Z inflating: build/lib/torch/include/ATen/ops/linalg_vecdot_native.h 2024-08-08T20:48:49.7433031Z inflating: build/lib/torch/include/ATen/ops/linalg_vecdot_ops.h 2024-08-08T20:48:49.7434100Z inflating: build/lib/torch/include/ATen/ops/linalg_vector_norm.h 2024-08-08T20:48:49.7435800Z inflating: build/lib/torch/include/ATen/ops/linalg_vector_norm_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7436518Z inflating: build/lib/torch/include/ATen/ops/linalg_vector_norm_cpu_dispatch.h 2024-08-08T20:48:49.7437528Z inflating: build/lib/torch/include/ATen/ops/linalg_vector_norm_cuda_dispatch.h 2024-08-08T20:48:49.7438434Z inflating: build/lib/torch/include/ATen/ops/linalg_vector_norm_meta.h 2024-08-08T20:48:49.7439574Z inflating: build/lib/torch/include/ATen/ops/linalg_vector_norm_meta_dispatch.h 2024-08-08T20:48:49.7440185Z inflating: build/lib/torch/include/ATen/ops/linalg_vector_norm_native.h 2024-08-08T20:48:49.7441221Z inflating: build/lib/torch/include/ATen/ops/linalg_vector_norm_ops.h 2024-08-08T20:48:49.7442074Z inflating: build/lib/torch/include/ATen/ops/linear.h 2024-08-08T20:48:49.7443168Z inflating: build/lib/torch/include/ATen/ops/linear_backward.h 2024-08-08T20:48:49.7444682Z inflating: build/lib/torch/include/ATen/ops/linear_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7445245Z inflating: build/lib/torch/include/ATen/ops/linear_backward_native.h 2024-08-08T20:48:49.7446100Z inflating: build/lib/torch/include/ATen/ops/linear_backward_ops.h 2024-08-08T20:48:49.7447565Z inflating: build/lib/torch/include/ATen/ops/linear_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7448436Z inflating: build/lib/torch/include/ATen/ops/linear_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7448837Z inflating: build/lib/torch/include/ATen/ops/linear_native.h 2024-08-08T20:48:49.7450341Z inflating: build/lib/torch/include/ATen/ops/linear_ops.h 2024-08-08T20:48:49.7451434Z inflating: build/lib/torch/include/ATen/ops/linspace.h 2024-08-08T20:48:49.7453028Z inflating: build/lib/torch/include/ATen/ops/linspace_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7453541Z inflating: build/lib/torch/include/ATen/ops/linspace_cpu_dispatch.h 2024-08-08T20:48:49.7454619Z inflating: build/lib/torch/include/ATen/ops/linspace_cuda_dispatch.h 2024-08-08T20:48:49.7455546Z inflating: build/lib/torch/include/ATen/ops/linspace_meta_dispatch.h 2024-08-08T20:48:49.7456577Z inflating: build/lib/torch/include/ATen/ops/linspace_native.h 2024-08-08T20:48:49.7458199Z inflating: build/lib/torch/include/ATen/ops/linspace_ops.h 2024-08-08T20:48:49.7459071Z inflating: build/lib/torch/include/ATen/ops/log.h 2024-08-08T20:48:49.7460630Z inflating: build/lib/torch/include/ATen/ops/log10.h 2024-08-08T20:48:49.7461908Z inflating: build/lib/torch/include/ATen/ops/log10_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7462735Z inflating: build/lib/torch/include/ATen/ops/log10_cpu_dispatch.h 2024-08-08T20:48:49.7463593Z inflating: build/lib/torch/include/ATen/ops/log10_cuda_dispatch.h 2024-08-08T20:48:49.7464203Z inflating: build/lib/torch/include/ATen/ops/log10_meta.h 2024-08-08T20:48:49.7465430Z inflating: build/lib/torch/include/ATen/ops/log10_meta_dispatch.h 2024-08-08T20:48:49.7466294Z inflating: build/lib/torch/include/ATen/ops/log10_native.h 2024-08-08T20:48:49.7467297Z inflating: build/lib/torch/include/ATen/ops/log10_ops.h 2024-08-08T20:48:49.7468235Z inflating: build/lib/torch/include/ATen/ops/log1p.h 2024-08-08T20:48:49.7469867Z inflating: build/lib/torch/include/ATen/ops/log1p_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7470523Z inflating: build/lib/torch/include/ATen/ops/log1p_cpu_dispatch.h 2024-08-08T20:48:49.7471262Z inflating: build/lib/torch/include/ATen/ops/log1p_cuda_dispatch.h 2024-08-08T20:48:49.7472203Z inflating: build/lib/torch/include/ATen/ops/log1p_meta.h 2024-08-08T20:48:49.7473280Z inflating: build/lib/torch/include/ATen/ops/log1p_meta_dispatch.h 2024-08-08T20:48:49.7474195Z inflating: build/lib/torch/include/ATen/ops/log1p_native.h 2024-08-08T20:48:49.7475207Z inflating: build/lib/torch/include/ATen/ops/log1p_ops.h 2024-08-08T20:48:49.7476125Z inflating: build/lib/torch/include/ATen/ops/log2.h 2024-08-08T20:48:49.7477732Z inflating: build/lib/torch/include/ATen/ops/log2_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7478405Z inflating: build/lib/torch/include/ATen/ops/log2_cpu_dispatch.h 2024-08-08T20:48:49.7479148Z inflating: build/lib/torch/include/ATen/ops/log2_cuda_dispatch.h 2024-08-08T20:48:49.7480077Z inflating: build/lib/torch/include/ATen/ops/log2_meta.h 2024-08-08T20:48:49.7481152Z inflating: build/lib/torch/include/ATen/ops/log2_meta_dispatch.h 2024-08-08T20:48:49.7481889Z inflating: build/lib/torch/include/ATen/ops/log2_native.h 2024-08-08T20:48:49.7483158Z inflating: build/lib/torch/include/ATen/ops/log2_ops.h 2024-08-08T20:48:49.7485502Z inflating: build/lib/torch/include/ATen/ops/log_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7486116Z inflating: build/lib/torch/include/ATen/ops/log_cpu_dispatch.h 2024-08-08T20:48:49.7486999Z inflating: build/lib/torch/include/ATen/ops/log_cuda_dispatch.h 2024-08-08T20:48:49.7487595Z inflating: build/lib/torch/include/ATen/ops/log_meta.h 2024-08-08T20:48:49.7488765Z inflating: build/lib/torch/include/ATen/ops/log_meta_dispatch.h 2024-08-08T20:48:49.7489636Z inflating: build/lib/torch/include/ATen/ops/log_native.h 2024-08-08T20:48:49.7490683Z inflating: build/lib/torch/include/ATen/ops/log_normal.h 2024-08-08T20:48:49.7492141Z inflating: build/lib/torch/include/ATen/ops/log_normal_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7492709Z inflating: build/lib/torch/include/ATen/ops/log_normal_cpu_dispatch.h 2024-08-08T20:48:49.7493320Z inflating: build/lib/torch/include/ATen/ops/log_normal_cuda_dispatch.h 2024-08-08T20:48:49.7494365Z inflating: build/lib/torch/include/ATen/ops/log_normal_meta_dispatch.h 2024-08-08T20:48:49.7495175Z inflating: build/lib/torch/include/ATen/ops/log_normal_native.h 2024-08-08T20:48:49.7496327Z inflating: build/lib/torch/include/ATen/ops/log_normal_ops.h 2024-08-08T20:48:49.7497407Z inflating: build/lib/torch/include/ATen/ops/log_ops.h 2024-08-08T20:48:49.7498290Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid.h 2024-08-08T20:48:49.7499444Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_backward.h 2024-08-08T20:48:49.7500641Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_backward_cpu_dispatch.h 2024-08-08T20:48:49.7501557Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_backward_cuda_dispatch.h 2024-08-08T20:48:49.7503262Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_backward_native.h 2024-08-08T20:48:49.7503953Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_backward_ops.h 2024-08-08T20:48:49.7505276Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7505887Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_forward.h 2024-08-08T20:48:49.7506996Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_forward_cpu_dispatch.h 2024-08-08T20:48:49.7508107Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_forward_cuda_dispatch.h 2024-08-08T20:48:49.7509190Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_forward_native.h 2024-08-08T20:48:49.7510485Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_forward_ops.h 2024-08-08T20:48:49.7511027Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_native.h 2024-08-08T20:48:49.7511991Z inflating: build/lib/torch/include/ATen/ops/log_sigmoid_ops.h 2024-08-08T20:48:49.7513037Z inflating: build/lib/torch/include/ATen/ops/log_softmax.h 2024-08-08T20:48:49.7515017Z inflating: build/lib/torch/include/ATen/ops/log_softmax_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7515860Z inflating: build/lib/torch/include/ATen/ops/log_softmax_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7516869Z inflating: build/lib/torch/include/ATen/ops/log_softmax_native.h 2024-08-08T20:48:49.7517980Z inflating: build/lib/torch/include/ATen/ops/log_softmax_ops.h 2024-08-08T20:48:49.7518897Z inflating: build/lib/torch/include/ATen/ops/logaddexp.h 2024-08-08T20:48:49.7519934Z inflating: build/lib/torch/include/ATen/ops/logaddexp2.h 2024-08-08T20:48:49.7521243Z inflating: build/lib/torch/include/ATen/ops/logaddexp2_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7522005Z inflating: build/lib/torch/include/ATen/ops/logaddexp2_cpu_dispatch.h 2024-08-08T20:48:49.7523176Z inflating: build/lib/torch/include/ATen/ops/logaddexp2_cuda_dispatch.h 2024-08-08T20:48:49.7524207Z inflating: build/lib/torch/include/ATen/ops/logaddexp2_meta.h 2024-08-08T20:48:49.7525296Z inflating: build/lib/torch/include/ATen/ops/logaddexp2_meta_dispatch.h 2024-08-08T20:48:49.7526115Z inflating: build/lib/torch/include/ATen/ops/logaddexp2_native.h 2024-08-08T20:48:49.7527176Z inflating: build/lib/torch/include/ATen/ops/logaddexp2_ops.h 2024-08-08T20:48:49.7528744Z inflating: build/lib/torch/include/ATen/ops/logaddexp_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7529491Z inflating: build/lib/torch/include/ATen/ops/logaddexp_cpu_dispatch.h 2024-08-08T20:48:49.7530095Z inflating: build/lib/torch/include/ATen/ops/logaddexp_cuda_dispatch.h 2024-08-08T20:48:49.7531037Z inflating: build/lib/torch/include/ATen/ops/logaddexp_meta.h 2024-08-08T20:48:49.7532161Z inflating: build/lib/torch/include/ATen/ops/logaddexp_meta_dispatch.h 2024-08-08T20:48:49.7532997Z inflating: build/lib/torch/include/ATen/ops/logaddexp_native.h 2024-08-08T20:48:49.7533989Z inflating: build/lib/torch/include/ATen/ops/logaddexp_ops.h 2024-08-08T20:48:49.7534964Z inflating: build/lib/torch/include/ATen/ops/logcumsumexp.h 2024-08-08T20:48:49.7536427Z inflating: build/lib/torch/include/ATen/ops/logcumsumexp_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7537197Z inflating: build/lib/torch/include/ATen/ops/logcumsumexp_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7538030Z inflating: build/lib/torch/include/ATen/ops/logcumsumexp_native.h 2024-08-08T20:48:49.7539103Z inflating: build/lib/torch/include/ATen/ops/logcumsumexp_ops.h 2024-08-08T20:48:49.7539953Z inflating: build/lib/torch/include/ATen/ops/logdet.h 2024-08-08T20:48:49.7541350Z inflating: build/lib/torch/include/ATen/ops/logdet_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7542071Z inflating: build/lib/torch/include/ATen/ops/logdet_native.h 2024-08-08T20:48:49.7543121Z inflating: build/lib/torch/include/ATen/ops/logdet_ops.h 2024-08-08T20:48:49.7543984Z inflating: build/lib/torch/include/ATen/ops/logical_and.h 2024-08-08T20:48:49.7545395Z inflating: build/lib/torch/include/ATen/ops/logical_and_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7545982Z inflating: build/lib/torch/include/ATen/ops/logical_and_cpu_dispatch.h 2024-08-08T20:48:49.7546690Z inflating: build/lib/torch/include/ATen/ops/logical_and_cuda_dispatch.h 2024-08-08T20:48:49.7547699Z inflating: build/lib/torch/include/ATen/ops/logical_and_native.h 2024-08-08T20:48:49.7548907Z inflating: build/lib/torch/include/ATen/ops/logical_and_ops.h 2024-08-08T20:48:49.7550611Z inflating: build/lib/torch/include/ATen/ops/logical_not.h 2024-08-08T20:48:49.7551628Z inflating: build/lib/torch/include/ATen/ops/logical_not_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7552525Z inflating: build/lib/torch/include/ATen/ops/logical_not_cpu_dispatch.h 2024-08-08T20:48:49.7553463Z inflating: build/lib/torch/include/ATen/ops/logical_not_cuda_dispatch.h 2024-08-08T20:48:49.7554456Z inflating: build/lib/torch/include/ATen/ops/logical_not_native.h 2024-08-08T20:48:49.7555539Z inflating: build/lib/torch/include/ATen/ops/logical_not_ops.h 2024-08-08T20:48:49.7556389Z inflating: build/lib/torch/include/ATen/ops/logical_or.h 2024-08-08T20:48:49.7557849Z inflating: build/lib/torch/include/ATen/ops/logical_or_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7559084Z inflating: build/lib/torch/include/ATen/ops/logical_or_cpu_dispatch.h 2024-08-08T20:48:49.7563393Z inflating: build/lib/torch/include/ATen/ops/logical_or_cuda_dispatch.h 2024-08-08T20:48:49.7563977Z inflating: build/lib/torch/include/ATen/ops/logical_or_native.h 2024-08-08T20:48:49.7565131Z inflating: build/lib/torch/include/ATen/ops/logical_or_ops.h 2024-08-08T20:48:49.7566110Z inflating: build/lib/torch/include/ATen/ops/logical_xor.h 2024-08-08T20:48:49.7567551Z inflating: build/lib/torch/include/ATen/ops/logical_xor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7568146Z inflating: build/lib/torch/include/ATen/ops/logical_xor_cpu_dispatch.h 2024-08-08T20:48:49.7569064Z inflating: build/lib/torch/include/ATen/ops/logical_xor_cuda_dispatch.h 2024-08-08T20:48:49.7570141Z inflating: build/lib/torch/include/ATen/ops/logical_xor_native.h 2024-08-08T20:48:49.7571183Z inflating: build/lib/torch/include/ATen/ops/logical_xor_ops.h 2024-08-08T20:48:49.7572119Z inflating: build/lib/torch/include/ATen/ops/logit.h 2024-08-08T20:48:49.7573194Z inflating: build/lib/torch/include/ATen/ops/logit_backward.h 2024-08-08T20:48:49.7574925Z inflating: build/lib/torch/include/ATen/ops/logit_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7576094Z inflating: build/lib/torch/include/ATen/ops/logit_backward_cpu_dispatch.h 2024-08-08T20:48:49.7576800Z inflating: build/lib/torch/include/ATen/ops/logit_backward_cuda_dispatch.h 2024-08-08T20:48:49.7577448Z inflating: build/lib/torch/include/ATen/ops/logit_backward_meta.h 2024-08-08T20:48:49.7578470Z inflating: build/lib/torch/include/ATen/ops/logit_backward_meta_dispatch.h 2024-08-08T20:48:49.7579345Z inflating: build/lib/torch/include/ATen/ops/logit_backward_native.h 2024-08-08T20:48:49.7579857Z inflating: build/lib/torch/include/ATen/ops/logit_backward_ops.h 2024-08-08T20:48:49.7580853Z inflating: build/lib/torch/include/ATen/ops/logit_cpu_dispatch.h 2024-08-08T20:48:49.7581925Z inflating: build/lib/torch/include/ATen/ops/logit_cuda_dispatch.h 2024-08-08T20:48:49.7583093Z inflating: build/lib/torch/include/ATen/ops/logit_meta_dispatch.h 2024-08-08T20:48:49.7583891Z inflating: build/lib/torch/include/ATen/ops/logit_native.h 2024-08-08T20:48:49.7585148Z inflating: build/lib/torch/include/ATen/ops/logit_ops.h 2024-08-08T20:48:49.7586844Z inflating: build/lib/torch/include/ATen/ops/logspace.h 2024-08-08T20:48:49.7588587Z inflating: build/lib/torch/include/ATen/ops/logspace_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7589384Z inflating: build/lib/torch/include/ATen/ops/logspace_cpu_dispatch.h 2024-08-08T20:48:49.7589899Z inflating: build/lib/torch/include/ATen/ops/logspace_cuda_dispatch.h 2024-08-08T20:48:49.7590550Z inflating: build/lib/torch/include/ATen/ops/logspace_meta_dispatch.h 2024-08-08T20:48:49.7591586Z inflating: build/lib/torch/include/ATen/ops/logspace_native.h 2024-08-08T20:48:49.7593164Z inflating: build/lib/torch/include/ATen/ops/logspace_ops.h 2024-08-08T20:48:49.7594085Z inflating: build/lib/torch/include/ATen/ops/logsumexp.h 2024-08-08T20:48:49.7595635Z inflating: build/lib/torch/include/ATen/ops/logsumexp_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7596945Z inflating: build/lib/torch/include/ATen/ops/logsumexp_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7597719Z inflating: build/lib/torch/include/ATen/ops/logsumexp_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7598447Z inflating: build/lib/torch/include/ATen/ops/logsumexp_native.h 2024-08-08T20:48:49.7599547Z inflating: build/lib/torch/include/ATen/ops/logsumexp_ops.h 2024-08-08T20:48:49.7600480Z inflating: build/lib/torch/include/ATen/ops/lshift.h 2024-08-08T20:48:49.7601905Z inflating: build/lib/torch/include/ATen/ops/lshift_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7602670Z inflating: build/lib/torch/include/ATen/ops/lshift_cpu_dispatch.h 2024-08-08T20:48:49.7603643Z inflating: build/lib/torch/include/ATen/ops/lshift_cuda_dispatch.h 2024-08-08T20:48:49.7604611Z inflating: build/lib/torch/include/ATen/ops/lshift_meta_dispatch.h 2024-08-08T20:48:49.7605612Z inflating: build/lib/torch/include/ATen/ops/lshift_native.h 2024-08-08T20:48:49.7606770Z inflating: build/lib/torch/include/ATen/ops/lshift_ops.h 2024-08-08T20:48:49.7607706Z inflating: build/lib/torch/include/ATen/ops/lstm.h 2024-08-08T20:48:49.7608726Z inflating: build/lib/torch/include/ATen/ops/lstm_cell.h 2024-08-08T20:48:49.7610145Z inflating: build/lib/torch/include/ATen/ops/lstm_cell_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7610913Z inflating: build/lib/torch/include/ATen/ops/lstm_cell_native.h 2024-08-08T20:48:49.7612145Z inflating: build/lib/torch/include/ATen/ops/lstm_cell_ops.h 2024-08-08T20:48:49.7613409Z inflating: build/lib/torch/include/ATen/ops/lstm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7614300Z inflating: build/lib/torch/include/ATen/ops/lstm_mps_backward.h 2024-08-08T20:48:49.7615890Z inflating: build/lib/torch/include/ATen/ops/lstm_mps_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7616510Z inflating: build/lib/torch/include/ATen/ops/lstm_mps_backward_native.h 2024-08-08T20:48:49.7617521Z inflating: build/lib/torch/include/ATen/ops/lstm_mps_backward_ops.h 2024-08-08T20:48:49.7618255Z inflating: build/lib/torch/include/ATen/ops/lstm_native.h 2024-08-08T20:48:49.7619809Z inflating: build/lib/torch/include/ATen/ops/lstm_ops.h 2024-08-08T20:48:49.7621444Z inflating: build/lib/torch/include/ATen/ops/lt.h 2024-08-08T20:48:49.7622585Z inflating: build/lib/torch/include/ATen/ops/lt_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7623420Z inflating: build/lib/torch/include/ATen/ops/lt_cpu_dispatch.h 2024-08-08T20:48:49.7624473Z inflating: build/lib/torch/include/ATen/ops/lt_cuda_dispatch.h 2024-08-08T20:48:49.7625539Z inflating: build/lib/torch/include/ATen/ops/lt_meta.h 2024-08-08T20:48:49.7626797Z inflating: build/lib/torch/include/ATen/ops/lt_meta_dispatch.h 2024-08-08T20:48:49.7627710Z inflating: build/lib/torch/include/ATen/ops/lt_native.h 2024-08-08T20:48:49.7628807Z inflating: build/lib/torch/include/ATen/ops/lt_ops.h 2024-08-08T20:48:49.7629909Z inflating: build/lib/torch/include/ATen/ops/lu_solve.h 2024-08-08T20:48:49.7631594Z inflating: build/lib/torch/include/ATen/ops/lu_solve_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7632176Z inflating: build/lib/torch/include/ATen/ops/lu_solve_native.h 2024-08-08T20:48:49.7633044Z inflating: build/lib/torch/include/ATen/ops/lu_solve_ops.h 2024-08-08T20:48:49.7633999Z inflating: build/lib/torch/include/ATen/ops/lu_unpack.h 2024-08-08T20:48:49.7635702Z inflating: build/lib/torch/include/ATen/ops/lu_unpack_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7636404Z inflating: build/lib/torch/include/ATen/ops/lu_unpack_cpu_dispatch.h 2024-08-08T20:48:49.7637225Z inflating: build/lib/torch/include/ATen/ops/lu_unpack_cuda_dispatch.h 2024-08-08T20:48:49.7638064Z inflating: build/lib/torch/include/ATen/ops/lu_unpack_meta.h 2024-08-08T20:48:49.7639147Z inflating: build/lib/torch/include/ATen/ops/lu_unpack_meta_dispatch.h 2024-08-08T20:48:49.7640057Z inflating: build/lib/torch/include/ATen/ops/lu_unpack_native.h 2024-08-08T20:48:49.7641161Z inflating: build/lib/torch/include/ATen/ops/lu_unpack_ops.h 2024-08-08T20:48:49.7641937Z inflating: build/lib/torch/include/ATen/ops/mH.h 2024-08-08T20:48:49.7643365Z inflating: build/lib/torch/include/ATen/ops/mH_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7644059Z inflating: build/lib/torch/include/ATen/ops/mH_native.h 2024-08-08T20:48:49.7644961Z inflating: build/lib/torch/include/ATen/ops/mH_ops.h 2024-08-08T20:48:49.7645814Z inflating: build/lib/torch/include/ATen/ops/mT.h 2024-08-08T20:48:49.7647261Z inflating: build/lib/torch/include/ATen/ops/mT_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7647874Z inflating: build/lib/torch/include/ATen/ops/mT_native.h 2024-08-08T20:48:49.7649101Z inflating: build/lib/torch/include/ATen/ops/margin_ranking_loss.h 2024-08-08T20:48:49.7650747Z inflating: build/lib/torch/include/ATen/ops/margin_ranking_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7651521Z inflating: build/lib/torch/include/ATen/ops/margin_ranking_loss_native.h 2024-08-08T20:48:49.7652131Z inflating: build/lib/torch/include/ATen/ops/margin_ranking_loss_ops.h 2024-08-08T20:48:49.7652987Z inflating: build/lib/torch/include/ATen/ops/masked_fill.h 2024-08-08T20:48:49.7654536Z inflating: build/lib/torch/include/ATen/ops/masked_fill_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7655558Z inflating: build/lib/torch/include/ATen/ops/masked_fill_cpu_dispatch.h 2024-08-08T20:48:49.7656715Z inflating: build/lib/torch/include/ATen/ops/masked_fill_cuda_dispatch.h 2024-08-08T20:48:49.7657677Z inflating: build/lib/torch/include/ATen/ops/masked_fill_meta_dispatch.h 2024-08-08T20:48:49.7658874Z inflating: build/lib/torch/include/ATen/ops/masked_fill_native.h 2024-08-08T20:48:49.7660203Z inflating: build/lib/torch/include/ATen/ops/masked_fill_ops.h 2024-08-08T20:48:49.7661311Z inflating: build/lib/torch/include/ATen/ops/masked_scatter.h 2024-08-08T20:48:49.7662336Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_backward.h 2024-08-08T20:48:49.7663852Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7664658Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_backward_native.h 2024-08-08T20:48:49.7665219Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_backward_ops.h 2024-08-08T20:48:49.7666097Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7667307Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_cpu_dispatch.h 2024-08-08T20:48:49.7668339Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_cuda_dispatch.h 2024-08-08T20:48:49.7669321Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_meta_dispatch.h 2024-08-08T20:48:49.7670116Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_native.h 2024-08-08T20:48:49.7671372Z inflating: build/lib/torch/include/ATen/ops/masked_scatter_ops.h 2024-08-08T20:48:49.7672336Z inflating: build/lib/torch/include/ATen/ops/masked_select.h 2024-08-08T20:48:49.7673318Z inflating: build/lib/torch/include/ATen/ops/masked_select_backward.h 2024-08-08T20:48:49.7674908Z inflating: build/lib/torch/include/ATen/ops/masked_select_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7675498Z inflating: build/lib/torch/include/ATen/ops/masked_select_backward_native.h 2024-08-08T20:48:49.7676222Z inflating: build/lib/torch/include/ATen/ops/masked_select_backward_ops.h 2024-08-08T20:48:49.7677302Z inflating: build/lib/torch/include/ATen/ops/masked_select_cpu_dispatch.h 2024-08-08T20:48:49.7678468Z inflating: build/lib/torch/include/ATen/ops/masked_select_cuda_dispatch.h 2024-08-08T20:48:49.7679521Z inflating: build/lib/torch/include/ATen/ops/masked_select_native.h 2024-08-08T20:48:49.7680610Z inflating: build/lib/torch/include/ATen/ops/masked_select_ops.h 2024-08-08T20:48:49.7681393Z inflating: build/lib/torch/include/ATen/ops/matmul.h 2024-08-08T20:48:49.7682525Z inflating: build/lib/torch/include/ATen/ops/matmul_backward.h 2024-08-08T20:48:49.7684088Z inflating: build/lib/torch/include/ATen/ops/matmul_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7684748Z inflating: build/lib/torch/include/ATen/ops/matmul_backward_native.h 2024-08-08T20:48:49.7685521Z inflating: build/lib/torch/include/ATen/ops/matmul_backward_ops.h 2024-08-08T20:48:49.7686803Z inflating: build/lib/torch/include/ATen/ops/matmul_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7688243Z inflating: build/lib/torch/include/ATen/ops/matmul_native.h 2024-08-08T20:48:49.7689314Z inflating: build/lib/torch/include/ATen/ops/matmul_ops.h 2024-08-08T20:48:49.7689905Z inflating: build/lib/torch/include/ATen/ops/matrix_H.h 2024-08-08T20:48:49.7691259Z inflating: build/lib/torch/include/ATen/ops/matrix_H_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7691876Z inflating: build/lib/torch/include/ATen/ops/matrix_H_native.h 2024-08-08T20:48:49.7692915Z inflating: build/lib/torch/include/ATen/ops/matrix_H_ops.h 2024-08-08T20:48:49.7693981Z inflating: build/lib/torch/include/ATen/ops/matrix_exp.h 2024-08-08T20:48:49.7695241Z inflating: build/lib/torch/include/ATen/ops/matrix_exp_backward.h 2024-08-08T20:48:49.7696721Z inflating: build/lib/torch/include/ATen/ops/matrix_exp_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7697452Z inflating: build/lib/torch/include/ATen/ops/matrix_exp_backward_native.h 2024-08-08T20:48:49.7698368Z inflating: build/lib/torch/include/ATen/ops/matrix_exp_backward_ops.h 2024-08-08T20:48:49.7699463Z inflating: build/lib/torch/include/ATen/ops/matrix_exp_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7699993Z inflating: build/lib/torch/include/ATen/ops/matrix_exp_native.h 2024-08-08T20:48:49.7700685Z inflating: build/lib/torch/include/ATen/ops/matrix_exp_ops.h 2024-08-08T20:48:49.7701698Z inflating: build/lib/torch/include/ATen/ops/matrix_power.h 2024-08-08T20:48:49.7703267Z inflating: build/lib/torch/include/ATen/ops/matrix_power_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7704039Z inflating: build/lib/torch/include/ATen/ops/matrix_power_native.h 2024-08-08T20:48:49.7705027Z inflating: build/lib/torch/include/ATen/ops/matrix_power_ops.h 2024-08-08T20:48:49.7706047Z inflating: build/lib/torch/include/ATen/ops/max.h 2024-08-08T20:48:49.7707733Z inflating: build/lib/torch/include/ATen/ops/max_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7708630Z inflating: build/lib/torch/include/ATen/ops/max_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7709391Z inflating: build/lib/torch/include/ATen/ops/max_cpu_dispatch.h 2024-08-08T20:48:49.7710289Z inflating: build/lib/torch/include/ATen/ops/max_cuda_dispatch.h 2024-08-08T20:48:49.7711185Z inflating: build/lib/torch/include/ATen/ops/max_meta.h 2024-08-08T20:48:49.7712471Z inflating: build/lib/torch/include/ATen/ops/max_meta_dispatch.h 2024-08-08T20:48:49.7713342Z inflating: build/lib/torch/include/ATen/ops/max_native.h 2024-08-08T20:48:49.7714564Z inflating: build/lib/torch/include/ATen/ops/max_ops.h 2024-08-08T20:48:49.7715564Z inflating: build/lib/torch/include/ATen/ops/max_pool1d.h 2024-08-08T20:48:49.7717028Z inflating: build/lib/torch/include/ATen/ops/max_pool1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7717513Z inflating: build/lib/torch/include/ATen/ops/max_pool1d_native.h 2024-08-08T20:48:49.7718153Z inflating: build/lib/torch/include/ATen/ops/max_pool1d_ops.h 2024-08-08T20:48:49.7719668Z inflating: build/lib/torch/include/ATen/ops/max_pool1d_with_indices.h 2024-08-08T20:48:49.7720717Z inflating: build/lib/torch/include/ATen/ops/max_pool1d_with_indices_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7721514Z inflating: build/lib/torch/include/ATen/ops/max_pool1d_with_indices_native.h 2024-08-08T20:48:49.7722668Z inflating: build/lib/torch/include/ATen/ops/max_pool1d_with_indices_ops.h 2024-08-08T20:48:49.7723724Z inflating: build/lib/torch/include/ATen/ops/max_pool2d.h 2024-08-08T20:48:49.7724790Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_backward.h 2024-08-08T20:48:49.7726351Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7727010Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_backward_native.h 2024-08-08T20:48:49.7727823Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_backward_ops.h 2024-08-08T20:48:49.7729275Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7729863Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_native.h 2024-08-08T20:48:49.7730789Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_ops.h 2024-08-08T20:48:49.7731910Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices.h 2024-08-08T20:48:49.7733062Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_backward.h 2024-08-08T20:48:49.7734777Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7735738Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_backward_cpu_dispatch.h 2024-08-08T20:48:49.7736804Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_backward_cuda_dispatch.h 2024-08-08T20:48:49.7737673Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_backward_meta.h 2024-08-08T20:48:49.7738671Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_backward_meta_dispatch.h 2024-08-08T20:48:49.7739481Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_backward_native.h 2024-08-08T20:48:49.7740264Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_backward_ops.h 2024-08-08T20:48:49.7741666Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7742330Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_cpu_dispatch.h 2024-08-08T20:48:49.7743290Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_cuda_dispatch.h 2024-08-08T20:48:49.7744423Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_meta.h 2024-08-08T20:48:49.7745543Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_meta_dispatch.h 2024-08-08T20:48:49.7746380Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_native.h 2024-08-08T20:48:49.7747533Z inflating: build/lib/torch/include/ATen/ops/max_pool2d_with_indices_ops.h 2024-08-08T20:48:49.7748352Z inflating: build/lib/torch/include/ATen/ops/max_pool3d.h 2024-08-08T20:48:49.7749860Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7750640Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_native.h 2024-08-08T20:48:49.7751245Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_ops.h 2024-08-08T20:48:49.7752532Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices.h 2024-08-08T20:48:49.7754707Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices_backward.h 2024-08-08T20:48:49.7755540Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices_backward_cpu_dispatch.h 2024-08-08T20:48:49.7756627Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices_backward_cuda_dispatch.h 2024-08-08T20:48:49.7757627Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices_backward_native.h 2024-08-08T20:48:49.7759252Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices_backward_ops.h 2024-08-08T20:48:49.7760107Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices_cpu_dispatch.h 2024-08-08T20:48:49.7761177Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices_cuda_dispatch.h 2024-08-08T20:48:49.7762107Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices_native.h 2024-08-08T20:48:49.7763036Z inflating: build/lib/torch/include/ATen/ops/max_pool3d_with_indices_ops.h 2024-08-08T20:48:49.7764153Z inflating: build/lib/torch/include/ATen/ops/max_unpool2d.h 2024-08-08T20:48:49.7765247Z inflating: build/lib/torch/include/ATen/ops/max_unpool2d_cpu_dispatch.h 2024-08-08T20:48:49.7766229Z inflating: build/lib/torch/include/ATen/ops/max_unpool2d_cuda_dispatch.h 2024-08-08T20:48:49.7767158Z inflating: build/lib/torch/include/ATen/ops/max_unpool2d_native.h 2024-08-08T20:48:49.7768195Z inflating: build/lib/torch/include/ATen/ops/max_unpool2d_ops.h 2024-08-08T20:48:49.7769373Z inflating: build/lib/torch/include/ATen/ops/max_unpool3d.h 2024-08-08T20:48:49.7770518Z inflating: build/lib/torch/include/ATen/ops/max_unpool3d_cpu_dispatch.h 2024-08-08T20:48:49.7771650Z inflating: build/lib/torch/include/ATen/ops/max_unpool3d_cuda_dispatch.h 2024-08-08T20:48:49.7772433Z inflating: build/lib/torch/include/ATen/ops/max_unpool3d_native.h 2024-08-08T20:48:49.7773492Z inflating: build/lib/torch/include/ATen/ops/max_unpool3d_ops.h 2024-08-08T20:48:49.7774463Z inflating: build/lib/torch/include/ATen/ops/maximum.h 2024-08-08T20:48:49.7776123Z inflating: build/lib/torch/include/ATen/ops/maximum_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7776790Z inflating: build/lib/torch/include/ATen/ops/maximum_cpu_dispatch.h 2024-08-08T20:48:49.7777670Z inflating: build/lib/torch/include/ATen/ops/maximum_cuda_dispatch.h 2024-08-08T20:48:49.7778328Z inflating: build/lib/torch/include/ATen/ops/maximum_meta.h 2024-08-08T20:48:49.7779465Z inflating: build/lib/torch/include/ATen/ops/maximum_meta_dispatch.h 2024-08-08T20:48:49.7780548Z inflating: build/lib/torch/include/ATen/ops/maximum_native.h 2024-08-08T20:48:49.7781445Z inflating: build/lib/torch/include/ATen/ops/maximum_ops.h 2024-08-08T20:48:49.7782430Z inflating: build/lib/torch/include/ATen/ops/mean.h 2024-08-08T20:48:49.7783785Z inflating: build/lib/torch/include/ATen/ops/mean_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7785096Z inflating: build/lib/torch/include/ATen/ops/mean_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7785915Z inflating: build/lib/torch/include/ATen/ops/mean_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7786775Z inflating: build/lib/torch/include/ATen/ops/mean_cpu_dispatch.h 2024-08-08T20:48:49.7787722Z inflating: build/lib/torch/include/ATen/ops/mean_cuda_dispatch.h 2024-08-08T20:48:49.7788566Z inflating: build/lib/torch/include/ATen/ops/mean_meta.h 2024-08-08T20:48:49.7789713Z inflating: build/lib/torch/include/ATen/ops/mean_meta_dispatch.h 2024-08-08T20:48:49.7790617Z inflating: build/lib/torch/include/ATen/ops/mean_native.h 2024-08-08T20:48:49.7791781Z inflating: build/lib/torch/include/ATen/ops/mean_ops.h 2024-08-08T20:48:49.7793047Z inflating: build/lib/torch/include/ATen/ops/median.h 2024-08-08T20:48:49.7794347Z inflating: build/lib/torch/include/ATen/ops/median_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7795538Z inflating: build/lib/torch/include/ATen/ops/median_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7796203Z inflating: build/lib/torch/include/ATen/ops/median_cpu_dispatch.h 2024-08-08T20:48:49.7797063Z inflating: build/lib/torch/include/ATen/ops/median_cuda_dispatch.h 2024-08-08T20:48:49.7797686Z inflating: build/lib/torch/include/ATen/ops/median_native.h 2024-08-08T20:48:49.7798858Z inflating: build/lib/torch/include/ATen/ops/median_ops.h 2024-08-08T20:48:49.7799681Z inflating: build/lib/torch/include/ATen/ops/meshgrid.h 2024-08-08T20:48:49.7801101Z inflating: build/lib/torch/include/ATen/ops/meshgrid_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7801940Z inflating: build/lib/torch/include/ATen/ops/meshgrid_native.h 2024-08-08T20:48:49.7803174Z inflating: build/lib/torch/include/ATen/ops/meshgrid_ops.h 2024-08-08T20:48:49.7804205Z inflating: build/lib/torch/include/ATen/ops/min.h 2024-08-08T20:48:49.7805740Z inflating: build/lib/torch/include/ATen/ops/min_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7806442Z inflating: build/lib/torch/include/ATen/ops/min_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7806942Z inflating: build/lib/torch/include/ATen/ops/min_cpu_dispatch.h 2024-08-08T20:48:49.7808066Z inflating: build/lib/torch/include/ATen/ops/min_cuda_dispatch.h 2024-08-08T20:48:49.7809087Z inflating: build/lib/torch/include/ATen/ops/min_meta.h 2024-08-08T20:48:49.7810401Z inflating: build/lib/torch/include/ATen/ops/min_meta_dispatch.h 2024-08-08T20:48:49.7811360Z inflating: build/lib/torch/include/ATen/ops/min_native.h 2024-08-08T20:48:49.7812659Z inflating: build/lib/torch/include/ATen/ops/min_ops.h 2024-08-08T20:48:49.7813839Z inflating: build/lib/torch/include/ATen/ops/minimum.h 2024-08-08T20:48:49.7815493Z inflating: build/lib/torch/include/ATen/ops/minimum_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7816171Z inflating: build/lib/torch/include/ATen/ops/minimum_cpu_dispatch.h 2024-08-08T20:48:49.7816936Z inflating: build/lib/torch/include/ATen/ops/minimum_cuda_dispatch.h 2024-08-08T20:48:49.7817843Z inflating: build/lib/torch/include/ATen/ops/minimum_meta.h 2024-08-08T20:48:49.7818788Z inflating: build/lib/torch/include/ATen/ops/minimum_meta_dispatch.h 2024-08-08T20:48:49.7819837Z inflating: build/lib/torch/include/ATen/ops/minimum_native.h 2024-08-08T20:48:49.7821647Z inflating: build/lib/torch/include/ATen/ops/minimum_ops.h 2024-08-08T20:48:49.7822429Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm.h 2024-08-08T20:48:49.7823863Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm_backward.h 2024-08-08T20:48:49.7825353Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7826028Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm_backward_cuda_dispatch.h 2024-08-08T20:48:49.7826910Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm_backward_native.h 2024-08-08T20:48:49.7828216Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm_backward_ops.h 2024-08-08T20:48:49.7829513Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7830523Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm_cuda_dispatch.h 2024-08-08T20:48:49.7831338Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm_native.h 2024-08-08T20:48:49.7832330Z inflating: build/lib/torch/include/ATen/ops/miopen_batch_norm_ops.h 2024-08-08T20:48:49.7833657Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution.h 2024-08-08T20:48:49.7835012Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_add_relu.h 2024-08-08T20:48:49.7836108Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_add_relu_cuda_dispatch.h 2024-08-08T20:48:49.7837046Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_add_relu_native.h 2024-08-08T20:48:49.7837981Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_add_relu_ops.h 2024-08-08T20:48:49.7839607Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7840479Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_cuda_dispatch.h 2024-08-08T20:48:49.7841180Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_native.h 2024-08-08T20:48:49.7842174Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_ops.h 2024-08-08T20:48:49.7843330Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_relu.h 2024-08-08T20:48:49.7844479Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_relu_cuda_dispatch.h 2024-08-08T20:48:49.7845314Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_relu_native.h 2024-08-08T20:48:49.7846313Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_relu_ops.h 2024-08-08T20:48:49.7847765Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_transpose.h 2024-08-08T20:48:49.7849400Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_transpose_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7850405Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_transpose_cuda_dispatch.h 2024-08-08T20:48:49.7851185Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_transpose_native.h 2024-08-08T20:48:49.7852050Z inflating: build/lib/torch/include/ATen/ops/miopen_convolution_transpose_ops.h 2024-08-08T20:48:49.7853449Z inflating: build/lib/torch/include/ATen/ops/miopen_depthwise_convolution.h 2024-08-08T20:48:49.7854817Z inflating: build/lib/torch/include/ATen/ops/miopen_depthwise_convolution_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7855910Z inflating: build/lib/torch/include/ATen/ops/miopen_depthwise_convolution_cuda_dispatch.h 2024-08-08T20:48:49.7856812Z inflating: build/lib/torch/include/ATen/ops/miopen_depthwise_convolution_native.h 2024-08-08T20:48:49.7857876Z inflating: build/lib/torch/include/ATen/ops/miopen_depthwise_convolution_ops.h 2024-08-08T20:48:49.7859165Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn.h 2024-08-08T20:48:49.7860623Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn_backward.h 2024-08-08T20:48:49.7862161Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7863188Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn_backward_cuda_dispatch.h 2024-08-08T20:48:49.7864104Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn_backward_native.h 2024-08-08T20:48:49.7865287Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn_backward_ops.h 2024-08-08T20:48:49.7866587Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7867480Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn_cuda_dispatch.h 2024-08-08T20:48:49.7868241Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn_native.h 2024-08-08T20:48:49.7869482Z inflating: build/lib/torch/include/ATen/ops/miopen_rnn_ops.h 2024-08-08T20:48:49.7870326Z inflating: build/lib/torch/include/ATen/ops/mish.h 2024-08-08T20:48:49.7871320Z inflating: build/lib/torch/include/ATen/ops/mish_backward.h 2024-08-08T20:48:49.7872852Z inflating: build/lib/torch/include/ATen/ops/mish_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7873544Z inflating: build/lib/torch/include/ATen/ops/mish_backward_cpu_dispatch.h 2024-08-08T20:48:49.7874128Z inflating: build/lib/torch/include/ATen/ops/mish_backward_cuda_dispatch.h 2024-08-08T20:48:49.7875276Z inflating: build/lib/torch/include/ATen/ops/mish_backward_native.h 2024-08-08T20:48:49.7876029Z inflating: build/lib/torch/include/ATen/ops/mish_backward_ops.h 2024-08-08T20:48:49.7877441Z inflating: build/lib/torch/include/ATen/ops/mish_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7878271Z inflating: build/lib/torch/include/ATen/ops/mish_cpu_dispatch.h 2024-08-08T20:48:49.7879397Z inflating: build/lib/torch/include/ATen/ops/mish_cuda_dispatch.h 2024-08-08T20:48:49.7880063Z inflating: build/lib/torch/include/ATen/ops/mish_meta.h 2024-08-08T20:48:49.7881294Z inflating: build/lib/torch/include/ATen/ops/mish_meta_dispatch.h 2024-08-08T20:48:49.7882279Z inflating: build/lib/torch/include/ATen/ops/mish_native.h 2024-08-08T20:48:49.7883314Z inflating: build/lib/torch/include/ATen/ops/mish_ops.h 2024-08-08T20:48:49.7884485Z inflating: build/lib/torch/include/ATen/ops/mkldnn_adaptive_avg_pool2d.h 2024-08-08T20:48:49.7885657Z inflating: build/lib/torch/include/ATen/ops/mkldnn_adaptive_avg_pool2d_backward.h 2024-08-08T20:48:49.7887103Z inflating: build/lib/torch/include/ATen/ops/mkldnn_adaptive_avg_pool2d_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7888038Z inflating: build/lib/torch/include/ATen/ops/mkldnn_adaptive_avg_pool2d_backward_native.h 2024-08-08T20:48:49.7888898Z inflating: build/lib/torch/include/ATen/ops/mkldnn_adaptive_avg_pool2d_backward_ops.h 2024-08-08T20:48:49.7889688Z inflating: build/lib/torch/include/ATen/ops/mkldnn_adaptive_avg_pool2d_native.h 2024-08-08T20:48:49.7891283Z inflating: build/lib/torch/include/ATen/ops/mkldnn_adaptive_avg_pool2d_ops.h 2024-08-08T20:48:49.7892481Z inflating: build/lib/torch/include/ATen/ops/mkldnn_convolution.h 2024-08-08T20:48:49.7894117Z inflating: build/lib/torch/include/ATen/ops/mkldnn_convolution_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7894811Z inflating: build/lib/torch/include/ATen/ops/mkldnn_convolution_native.h 2024-08-08T20:48:49.7895702Z inflating: build/lib/torch/include/ATen/ops/mkldnn_convolution_ops.h 2024-08-08T20:48:49.7896713Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear.h 2024-08-08T20:48:49.7897852Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward.h 2024-08-08T20:48:49.7899366Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7900129Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_input.h 2024-08-08T20:48:49.7901488Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_input_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7902300Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_input_native.h 2024-08-08T20:48:49.7903386Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_input_ops.h 2024-08-08T20:48:49.7904095Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_native.h 2024-08-08T20:48:49.7905101Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_ops.h 2024-08-08T20:48:49.7906239Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_weights.h 2024-08-08T20:48:49.7907727Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_weights_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7908534Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_weights_native.h 2024-08-08T20:48:49.7909239Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_backward_weights_ops.h 2024-08-08T20:48:49.7910048Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7910876Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_native.h 2024-08-08T20:48:49.7912033Z inflating: build/lib/torch/include/ATen/ops/mkldnn_linear_ops.h 2024-08-08T20:48:49.7913248Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool2d.h 2024-08-08T20:48:49.7914547Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool2d_backward.h 2024-08-08T20:48:49.7915899Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool2d_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7916645Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool2d_backward_native.h 2024-08-08T20:48:49.7917588Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool2d_backward_ops.h 2024-08-08T20:48:49.7919101Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool2d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7919756Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool2d_native.h 2024-08-08T20:48:49.7920574Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool2d_ops.h 2024-08-08T20:48:49.7921624Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool3d.h 2024-08-08T20:48:49.7922694Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool3d_backward.h 2024-08-08T20:48:49.7924481Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool3d_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7925239Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool3d_backward_native.h 2024-08-08T20:48:49.7926180Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool3d_backward_ops.h 2024-08-08T20:48:49.7927622Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool3d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7928301Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool3d_native.h 2024-08-08T20:48:49.7929227Z inflating: build/lib/torch/include/ATen/ops/mkldnn_max_pool3d_ops.h 2024-08-08T20:48:49.7930653Z inflating: build/lib/torch/include/ATen/ops/mkldnn_reorder_conv2d_weight.h 2024-08-08T20:48:49.7932262Z inflating: build/lib/torch/include/ATen/ops/mkldnn_reorder_conv2d_weight_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7933074Z inflating: build/lib/torch/include/ATen/ops/mkldnn_reorder_conv2d_weight_native.h 2024-08-08T20:48:49.7933749Z inflating: build/lib/torch/include/ATen/ops/mkldnn_reorder_conv2d_weight_ops.h 2024-08-08T20:48:49.7934784Z inflating: build/lib/torch/include/ATen/ops/mkldnn_reorder_conv3d_weight.h 2024-08-08T20:48:49.7936237Z inflating: build/lib/torch/include/ATen/ops/mkldnn_reorder_conv3d_weight_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7936908Z inflating: build/lib/torch/include/ATen/ops/mkldnn_reorder_conv3d_weight_native.h 2024-08-08T20:48:49.7938073Z inflating: build/lib/torch/include/ATen/ops/mkldnn_reorder_conv3d_weight_ops.h 2024-08-08T20:48:49.7939104Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer.h 2024-08-08T20:48:49.7940740Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer_backward.h 2024-08-08T20:48:49.7941893Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7942803Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer_backward_cpu_dispatch.h 2024-08-08T20:48:49.7943888Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer_backward_native.h 2024-08-08T20:48:49.7945270Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer_backward_ops.h 2024-08-08T20:48:49.7946671Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7947605Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer_cpu_dispatch.h 2024-08-08T20:48:49.7948570Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer_native.h 2024-08-08T20:48:49.7949759Z inflating: build/lib/torch/include/ATen/ops/mkldnn_rnn_layer_ops.h 2024-08-08T20:48:49.7950585Z inflating: build/lib/torch/include/ATen/ops/mm.h 2024-08-08T20:48:49.7952228Z inflating: build/lib/torch/include/ATen/ops/mm_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7952850Z inflating: build/lib/torch/include/ATen/ops/mm_cpu_dispatch.h 2024-08-08T20:48:49.7953906Z inflating: build/lib/torch/include/ATen/ops/mm_cuda_dispatch.h 2024-08-08T20:48:49.7954564Z inflating: build/lib/torch/include/ATen/ops/mm_meta.h 2024-08-08T20:48:49.7955672Z inflating: build/lib/torch/include/ATen/ops/mm_meta_dispatch.h 2024-08-08T20:48:49.7956636Z inflating: build/lib/torch/include/ATen/ops/mm_native.h 2024-08-08T20:48:49.7957485Z inflating: build/lib/torch/include/ATen/ops/mm_ops.h 2024-08-08T20:48:49.7959150Z inflating: build/lib/torch/include/ATen/ops/mode.h 2024-08-08T20:48:49.7960989Z inflating: build/lib/torch/include/ATen/ops/mode_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7961966Z inflating: build/lib/torch/include/ATen/ops/mode_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7962846Z inflating: build/lib/torch/include/ATen/ops/mode_cpu_dispatch.h 2024-08-08T20:48:49.7963962Z inflating: build/lib/torch/include/ATen/ops/mode_cuda_dispatch.h 2024-08-08T20:48:49.7964890Z inflating: build/lib/torch/include/ATen/ops/mode_native.h 2024-08-08T20:48:49.7966000Z inflating: build/lib/torch/include/ATen/ops/mode_ops.h 2024-08-08T20:48:49.7966973Z inflating: build/lib/torch/include/ATen/ops/moveaxis.h 2024-08-08T20:48:49.7968480Z inflating: build/lib/torch/include/ATen/ops/moveaxis_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7969230Z inflating: build/lib/torch/include/ATen/ops/moveaxis_native.h 2024-08-08T20:48:49.7970161Z inflating: build/lib/torch/include/ATen/ops/moveaxis_ops.h 2024-08-08T20:48:49.7971034Z inflating: build/lib/torch/include/ATen/ops/movedim.h 2024-08-08T20:48:49.7972508Z inflating: build/lib/torch/include/ATen/ops/movedim_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7973278Z inflating: build/lib/torch/include/ATen/ops/movedim_native.h 2024-08-08T20:48:49.7974160Z inflating: build/lib/torch/include/ATen/ops/movedim_ops.h 2024-08-08T20:48:49.7975716Z inflating: build/lib/torch/include/ATen/ops/mps_convolution_backward.h 2024-08-08T20:48:49.7977367Z inflating: build/lib/torch/include/ATen/ops/mps_convolution_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7978061Z inflating: build/lib/torch/include/ATen/ops/mps_convolution_backward_native.h 2024-08-08T20:48:49.7978983Z inflating: build/lib/torch/include/ATen/ops/mps_convolution_backward_ops.h 2024-08-08T20:48:49.7980324Z inflating: build/lib/torch/include/ATen/ops/mps_convolution_transpose_backward.h 2024-08-08T20:48:49.7981650Z inflating: build/lib/torch/include/ATen/ops/mps_convolution_transpose_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.7982607Z inflating: build/lib/torch/include/ATen/ops/mps_convolution_transpose_backward_native.h 2024-08-08T20:48:49.7984010Z inflating: build/lib/torch/include/ATen/ops/mps_convolution_transpose_backward_ops.h 2024-08-08T20:48:49.7984839Z inflating: build/lib/torch/include/ATen/ops/mse_loss.h 2024-08-08T20:48:49.7985780Z inflating: build/lib/torch/include/ATen/ops/mse_loss_backward.h 2024-08-08T20:48:49.7986932Z inflating: build/lib/torch/include/ATen/ops/mse_loss_backward_cpu_dispatch.h 2024-08-08T20:48:49.7987929Z inflating: build/lib/torch/include/ATen/ops/mse_loss_backward_cuda_dispatch.h 2024-08-08T20:48:49.7988840Z inflating: build/lib/torch/include/ATen/ops/mse_loss_backward_native.h 2024-08-08T20:48:49.7989916Z inflating: build/lib/torch/include/ATen/ops/mse_loss_backward_ops.h 2024-08-08T20:48:49.7991423Z inflating: build/lib/torch/include/ATen/ops/mse_loss_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.7992082Z inflating: build/lib/torch/include/ATen/ops/mse_loss_cpu_dispatch.h 2024-08-08T20:48:49.7992869Z inflating: build/lib/torch/include/ATen/ops/mse_loss_cuda_dispatch.h 2024-08-08T20:48:49.7994015Z inflating: build/lib/torch/include/ATen/ops/mse_loss_meta.h 2024-08-08T20:48:49.7995123Z inflating: build/lib/torch/include/ATen/ops/mse_loss_meta_dispatch.h 2024-08-08T20:48:49.7996139Z inflating: build/lib/torch/include/ATen/ops/mse_loss_native.h 2024-08-08T20:48:49.7997074Z inflating: build/lib/torch/include/ATen/ops/mse_loss_ops.h 2024-08-08T20:48:49.7997943Z inflating: build/lib/torch/include/ATen/ops/msort.h 2024-08-08T20:48:49.7999417Z inflating: build/lib/torch/include/ATen/ops/msort_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.7999922Z inflating: build/lib/torch/include/ATen/ops/msort_native.h 2024-08-08T20:48:49.8000632Z inflating: build/lib/torch/include/ATen/ops/msort_ops.h 2024-08-08T20:48:49.8001685Z inflating: build/lib/torch/include/ATen/ops/mul.h 2024-08-08T20:48:49.8002878Z inflating: build/lib/torch/include/ATen/ops/mul_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8004080Z inflating: build/lib/torch/include/ATen/ops/mul_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8004762Z inflating: build/lib/torch/include/ATen/ops/mul_cpu_dispatch.h 2024-08-08T20:48:49.8005841Z inflating: build/lib/torch/include/ATen/ops/mul_cuda_dispatch.h 2024-08-08T20:48:49.8006689Z inflating: build/lib/torch/include/ATen/ops/mul_meta.h 2024-08-08T20:48:49.8007807Z inflating: build/lib/torch/include/ATen/ops/mul_meta_dispatch.h 2024-08-08T20:48:49.8008759Z inflating: build/lib/torch/include/ATen/ops/mul_native.h 2024-08-08T20:48:49.8010278Z inflating: build/lib/torch/include/ATen/ops/mul_ops.h 2024-08-08T20:48:49.8011575Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss.h 2024-08-08T20:48:49.8012572Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss_backward.h 2024-08-08T20:48:49.8013801Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss_backward_cpu_dispatch.h 2024-08-08T20:48:49.8014795Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss_backward_cuda_dispatch.h 2024-08-08T20:48:49.8015802Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss_backward_native.h 2024-08-08T20:48:49.8016911Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss_backward_ops.h 2024-08-08T20:48:49.8017947Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss_cpu_dispatch.h 2024-08-08T20:48:49.8019042Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss_cuda_dispatch.h 2024-08-08T20:48:49.8019989Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss_native.h 2024-08-08T20:48:49.8021074Z inflating: build/lib/torch/include/ATen/ops/multi_margin_loss_ops.h 2024-08-08T20:48:49.8022146Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss.h 2024-08-08T20:48:49.8023241Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_backward.h 2024-08-08T20:48:49.8024389Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_backward_cpu_dispatch.h 2024-08-08T20:48:49.8025405Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_backward_cuda_dispatch.h 2024-08-08T20:48:49.8026387Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_backward_native.h 2024-08-08T20:48:49.8028284Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_backward_ops.h 2024-08-08T20:48:49.8030307Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8031402Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_forward.h 2024-08-08T20:48:49.8032669Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_forward_cpu_dispatch.h 2024-08-08T20:48:49.8033984Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_forward_cuda_dispatch.h 2024-08-08T20:48:49.8034837Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_forward_native.h 2024-08-08T20:48:49.8035973Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_forward_ops.h 2024-08-08T20:48:49.8036939Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_native.h 2024-08-08T20:48:49.8038116Z inflating: build/lib/torch/include/ATen/ops/multilabel_margin_loss_ops.h 2024-08-08T20:48:49.8038780Z inflating: build/lib/torch/include/ATen/ops/multinomial.h 2024-08-08T20:48:49.8039760Z inflating: build/lib/torch/include/ATen/ops/multinomial_cpu_dispatch.h 2024-08-08T20:48:49.8040742Z inflating: build/lib/torch/include/ATen/ops/multinomial_cuda_dispatch.h 2024-08-08T20:48:49.8041590Z inflating: build/lib/torch/include/ATen/ops/multinomial_native.h 2024-08-08T20:48:49.8042342Z inflating: build/lib/torch/include/ATen/ops/multinomial_ops.h 2024-08-08T20:48:49.8042958Z inflating: build/lib/torch/include/ATen/ops/multiply.h 2024-08-08T20:48:49.8044295Z inflating: build/lib/torch/include/ATen/ops/multiply_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8045032Z inflating: build/lib/torch/include/ATen/ops/multiply_native.h 2024-08-08T20:48:49.8046158Z inflating: build/lib/torch/include/ATen/ops/multiply_ops.h 2024-08-08T20:48:49.8046724Z inflating: build/lib/torch/include/ATen/ops/mv.h 2024-08-08T20:48:49.8048081Z inflating: build/lib/torch/include/ATen/ops/mv_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8048762Z inflating: build/lib/torch/include/ATen/ops/mv_native.h 2024-08-08T20:48:49.8049516Z inflating: build/lib/torch/include/ATen/ops/mv_ops.h 2024-08-08T20:48:49.8050323Z inflating: build/lib/torch/include/ATen/ops/mvlgamma.h 2024-08-08T20:48:49.8052102Z inflating: build/lib/torch/include/ATen/ops/mvlgamma_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8052978Z inflating: build/lib/torch/include/ATen/ops/mvlgamma_cpu_dispatch.h 2024-08-08T20:48:49.8053896Z inflating: build/lib/torch/include/ATen/ops/mvlgamma_cuda_dispatch.h 2024-08-08T20:48:49.8054653Z inflating: build/lib/torch/include/ATen/ops/mvlgamma_native.h 2024-08-08T20:48:49.8055689Z inflating: build/lib/torch/include/ATen/ops/mvlgamma_ops.h 2024-08-08T20:48:49.8056588Z inflating: build/lib/torch/include/ATen/ops/nan_to_num.h 2024-08-08T20:48:49.8058104Z inflating: build/lib/torch/include/ATen/ops/nan_to_num_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8059083Z inflating: build/lib/torch/include/ATen/ops/nan_to_num_cpu_dispatch.h 2024-08-08T20:48:49.8059792Z inflating: build/lib/torch/include/ATen/ops/nan_to_num_cuda_dispatch.h 2024-08-08T20:48:49.8060731Z inflating: build/lib/torch/include/ATen/ops/nan_to_num_native.h 2024-08-08T20:48:49.8062017Z inflating: build/lib/torch/include/ATen/ops/nan_to_num_ops.h 2024-08-08T20:48:49.8063087Z inflating: build/lib/torch/include/ATen/ops/nanmean.h 2024-08-08T20:48:49.8064546Z inflating: build/lib/torch/include/ATen/ops/nanmean_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8065061Z inflating: build/lib/torch/include/ATen/ops/nanmean_native.h 2024-08-08T20:48:49.8066122Z inflating: build/lib/torch/include/ATen/ops/nanmean_ops.h 2024-08-08T20:48:49.8067250Z inflating: build/lib/torch/include/ATen/ops/nanmedian.h 2024-08-08T20:48:49.8068704Z inflating: build/lib/torch/include/ATen/ops/nanmedian_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8069506Z inflating: build/lib/torch/include/ATen/ops/nanmedian_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8070338Z inflating: build/lib/torch/include/ATen/ops/nanmedian_cpu_dispatch.h 2024-08-08T20:48:49.8071242Z inflating: build/lib/torch/include/ATen/ops/nanmedian_cuda_dispatch.h 2024-08-08T20:48:49.8072207Z inflating: build/lib/torch/include/ATen/ops/nanmedian_native.h 2024-08-08T20:48:49.8073424Z inflating: build/lib/torch/include/ATen/ops/nanmedian_ops.h 2024-08-08T20:48:49.8074485Z inflating: build/lib/torch/include/ATen/ops/nanquantile.h 2024-08-08T20:48:49.8076062Z inflating: build/lib/torch/include/ATen/ops/nanquantile_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8076664Z inflating: build/lib/torch/include/ATen/ops/nanquantile_native.h 2024-08-08T20:48:49.8077414Z inflating: build/lib/torch/include/ATen/ops/nanquantile_ops.h 2024-08-08T20:48:49.8078630Z inflating: build/lib/torch/include/ATen/ops/nansum.h 2024-08-08T20:48:49.8079641Z inflating: build/lib/torch/include/ATen/ops/nansum_cpu_dispatch.h 2024-08-08T20:48:49.8080876Z inflating: build/lib/torch/include/ATen/ops/nansum_cuda_dispatch.h 2024-08-08T20:48:49.8081408Z inflating: build/lib/torch/include/ATen/ops/nansum_native.h 2024-08-08T20:48:49.8082449Z inflating: build/lib/torch/include/ATen/ops/nansum_ops.h 2024-08-08T20:48:49.8083516Z inflating: build/lib/torch/include/ATen/ops/narrow.h 2024-08-08T20:48:49.8085158Z inflating: build/lib/torch/include/ATen/ops/narrow_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8085940Z inflating: build/lib/torch/include/ATen/ops/narrow_copy.h 2024-08-08T20:48:49.8087586Z inflating: build/lib/torch/include/ATen/ops/narrow_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8088359Z inflating: build/lib/torch/include/ATen/ops/narrow_copy_cpu_dispatch.h 2024-08-08T20:48:49.8088920Z inflating: build/lib/torch/include/ATen/ops/narrow_copy_native.h 2024-08-08T20:48:49.8090123Z inflating: build/lib/torch/include/ATen/ops/narrow_copy_ops.h 2024-08-08T20:48:49.8091024Z inflating: build/lib/torch/include/ATen/ops/narrow_native.h 2024-08-08T20:48:49.8092075Z inflating: build/lib/torch/include/ATen/ops/narrow_ops.h 2024-08-08T20:48:49.8093106Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm.h 2024-08-08T20:48:49.8094416Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_backward.h 2024-08-08T20:48:49.8096952Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8097908Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_backward_cpu_dispatch.h 2024-08-08T20:48:49.8098832Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_backward_cuda_dispatch.h 2024-08-08T20:48:49.8099563Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_backward_native.h 2024-08-08T20:48:49.8100683Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_backward_ops.h 2024-08-08T20:48:49.8101760Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_cpu_dispatch.h 2024-08-08T20:48:49.8102795Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_cuda_dispatch.h 2024-08-08T20:48:49.8103757Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_native.h 2024-08-08T20:48:49.8104875Z inflating: build/lib/torch/include/ATen/ops/native_batch_norm_ops.h 2024-08-08T20:48:49.8105872Z inflating: build/lib/torch/include/ATen/ops/native_channel_shuffle.h 2024-08-08T20:48:49.8107427Z inflating: build/lib/torch/include/ATen/ops/native_channel_shuffle_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8108284Z inflating: build/lib/torch/include/ATen/ops/native_channel_shuffle_cpu_dispatch.h 2024-08-08T20:48:49.8109290Z inflating: build/lib/torch/include/ATen/ops/native_channel_shuffle_native.h 2024-08-08T20:48:49.8110036Z inflating: build/lib/torch/include/ATen/ops/native_channel_shuffle_ops.h 2024-08-08T20:48:49.8110608Z inflating: build/lib/torch/include/ATen/ops/native_dropout.h 2024-08-08T20:48:49.8111534Z inflating: build/lib/torch/include/ATen/ops/native_dropout_backward.h 2024-08-08T20:48:49.8112680Z inflating: build/lib/torch/include/ATen/ops/native_dropout_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8113571Z inflating: build/lib/torch/include/ATen/ops/native_dropout_backward_cpu_dispatch.h 2024-08-08T20:48:49.8114431Z inflating: build/lib/torch/include/ATen/ops/native_dropout_backward_cuda_dispatch.h 2024-08-08T20:48:49.8115459Z inflating: build/lib/torch/include/ATen/ops/native_dropout_backward_native.h 2024-08-08T20:48:49.8116500Z inflating: build/lib/torch/include/ATen/ops/native_dropout_backward_ops.h 2024-08-08T20:48:49.8117982Z inflating: build/lib/torch/include/ATen/ops/native_dropout_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8119115Z inflating: build/lib/torch/include/ATen/ops/native_dropout_cpu_dispatch.h 2024-08-08T20:48:49.8119869Z inflating: build/lib/torch/include/ATen/ops/native_dropout_cuda_dispatch.h 2024-08-08T20:48:49.8120691Z inflating: build/lib/torch/include/ATen/ops/native_dropout_native.h 2024-08-08T20:48:49.8121706Z inflating: build/lib/torch/include/ATen/ops/native_dropout_ops.h 2024-08-08T20:48:49.8122994Z inflating: build/lib/torch/include/ATen/ops/native_group_norm.h 2024-08-08T20:48:49.8124444Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_backward.h 2024-08-08T20:48:49.8125968Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8126898Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_backward_cpu_dispatch.h 2024-08-08T20:48:49.8127619Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_backward_cuda_dispatch.h 2024-08-08T20:48:49.8128845Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_backward_native.h 2024-08-08T20:48:49.8129967Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_backward_ops.h 2024-08-08T20:48:49.8131310Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8132304Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_cpu_dispatch.h 2024-08-08T20:48:49.8133272Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_cuda_dispatch.h 2024-08-08T20:48:49.8134125Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_native.h 2024-08-08T20:48:49.8135146Z inflating: build/lib/torch/include/ATen/ops/native_group_norm_ops.h 2024-08-08T20:48:49.8136397Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm.h 2024-08-08T20:48:49.8137905Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_backward.h 2024-08-08T20:48:49.8139545Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8140418Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_backward_cpu_dispatch.h 2024-08-08T20:48:49.8141488Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_backward_cuda_dispatch.h 2024-08-08T20:48:49.8142380Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_backward_native.h 2024-08-08T20:48:49.8143414Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_backward_ops.h 2024-08-08T20:48:49.8144855Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8145604Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_cpu_dispatch.h 2024-08-08T20:48:49.8146431Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_cuda_dispatch.h 2024-08-08T20:48:49.8147579Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_native.h 2024-08-08T20:48:49.8148597Z inflating: build/lib/torch/include/ATen/ops/native_layer_norm_ops.h 2024-08-08T20:48:49.8149572Z inflating: build/lib/torch/include/ATen/ops/native_norm.h 2024-08-08T20:48:49.8151072Z inflating: build/lib/torch/include/ATen/ops/native_norm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8151701Z inflating: build/lib/torch/include/ATen/ops/native_norm_native.h 2024-08-08T20:48:49.8152737Z inflating: build/lib/torch/include/ATen/ops/native_norm_ops.h 2024-08-08T20:48:49.8153662Z inflating: build/lib/torch/include/ATen/ops/ne.h 2024-08-08T20:48:49.8155226Z inflating: build/lib/torch/include/ATen/ops/ne_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8155832Z inflating: build/lib/torch/include/ATen/ops/ne_cpu_dispatch.h 2024-08-08T20:48:49.8156553Z inflating: build/lib/torch/include/ATen/ops/ne_cuda_dispatch.h 2024-08-08T20:48:49.8157675Z inflating: build/lib/torch/include/ATen/ops/ne_meta.h 2024-08-08T20:48:49.8159015Z inflating: build/lib/torch/include/ATen/ops/ne_meta_dispatch.h 2024-08-08T20:48:49.8160224Z inflating: build/lib/torch/include/ATen/ops/ne_native.h 2024-08-08T20:48:49.8161224Z inflating: build/lib/torch/include/ATen/ops/ne_ops.h 2024-08-08T20:48:49.8162099Z inflating: build/lib/torch/include/ATen/ops/neg.h 2024-08-08T20:48:49.8164045Z inflating: build/lib/torch/include/ATen/ops/neg_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8165065Z inflating: build/lib/torch/include/ATen/ops/neg_cpu_dispatch.h 2024-08-08T20:48:49.8166019Z inflating: build/lib/torch/include/ATen/ops/neg_cuda_dispatch.h 2024-08-08T20:48:49.8166778Z inflating: build/lib/torch/include/ATen/ops/neg_meta.h 2024-08-08T20:48:49.8167825Z inflating: build/lib/torch/include/ATen/ops/neg_meta_dispatch.h 2024-08-08T20:48:49.8168704Z inflating: build/lib/torch/include/ATen/ops/neg_native.h 2024-08-08T20:48:49.8169732Z inflating: build/lib/torch/include/ATen/ops/neg_ops.h 2024-08-08T20:48:49.8170712Z inflating: build/lib/torch/include/ATen/ops/negative.h 2024-08-08T20:48:49.8172438Z inflating: build/lib/torch/include/ATen/ops/negative_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8173204Z inflating: build/lib/torch/include/ATen/ops/negative_native.h 2024-08-08T20:48:49.8173955Z inflating: build/lib/torch/include/ATen/ops/negative_ops.h 2024-08-08T20:48:49.8174942Z inflating: build/lib/torch/include/ATen/ops/nested_to_padded_tensor.h 2024-08-08T20:48:49.8176744Z inflating: build/lib/torch/include/ATen/ops/nested_to_padded_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8177811Z inflating: build/lib/torch/include/ATen/ops/nested_to_padded_tensor_native.h 2024-08-08T20:48:49.8178658Z inflating: build/lib/torch/include/ATen/ops/nested_to_padded_tensor_ops.h 2024-08-08T20:48:49.8179363Z inflating: build/lib/torch/include/ATen/ops/new_empty.h 2024-08-08T20:48:49.8180605Z inflating: build/lib/torch/include/ATen/ops/new_empty_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8181402Z inflating: build/lib/torch/include/ATen/ops/new_empty_native.h 2024-08-08T20:48:49.8182165Z inflating: build/lib/torch/include/ATen/ops/new_empty_ops.h 2024-08-08T20:48:49.8183418Z inflating: build/lib/torch/include/ATen/ops/new_empty_strided.h 2024-08-08T20:48:49.8184860Z inflating: build/lib/torch/include/ATen/ops/new_empty_strided_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8186770Z inflating: build/lib/torch/include/ATen/ops/new_empty_strided_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8187767Z inflating: build/lib/torch/include/ATen/ops/new_empty_strided_native.h 2024-08-08T20:48:49.8188639Z inflating: build/lib/torch/include/ATen/ops/new_empty_strided_ops.h 2024-08-08T20:48:49.8189253Z inflating: build/lib/torch/include/ATen/ops/new_full.h 2024-08-08T20:48:49.8190888Z inflating: build/lib/torch/include/ATen/ops/new_full_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8191504Z inflating: build/lib/torch/include/ATen/ops/new_full_native.h 2024-08-08T20:48:49.8192239Z inflating: build/lib/torch/include/ATen/ops/new_full_ops.h 2024-08-08T20:48:49.8192886Z inflating: build/lib/torch/include/ATen/ops/new_ones.h 2024-08-08T20:48:49.8194513Z inflating: build/lib/torch/include/ATen/ops/new_ones_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8195262Z inflating: build/lib/torch/include/ATen/ops/new_ones_native.h 2024-08-08T20:48:49.8196209Z inflating: build/lib/torch/include/ATen/ops/new_ones_ops.h 2024-08-08T20:48:49.8197357Z inflating: build/lib/torch/include/ATen/ops/new_zeros.h 2024-08-08T20:48:49.8198965Z inflating: build/lib/torch/include/ATen/ops/new_zeros_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8199760Z inflating: build/lib/torch/include/ATen/ops/new_zeros_native.h 2024-08-08T20:48:49.8200634Z inflating: build/lib/torch/include/ATen/ops/new_zeros_ops.h 2024-08-08T20:48:49.8201405Z inflating: build/lib/torch/include/ATen/ops/nextafter.h 2024-08-08T20:48:49.8203703Z inflating: build/lib/torch/include/ATen/ops/nextafter_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8204652Z inflating: build/lib/torch/include/ATen/ops/nextafter_cpu_dispatch.h 2024-08-08T20:48:49.8205594Z inflating: build/lib/torch/include/ATen/ops/nextafter_cuda_dispatch.h 2024-08-08T20:48:49.8206344Z inflating: build/lib/torch/include/ATen/ops/nextafter_meta.h 2024-08-08T20:48:49.8207290Z inflating: build/lib/torch/include/ATen/ops/nextafter_meta_dispatch.h 2024-08-08T20:48:49.8208049Z inflating: build/lib/torch/include/ATen/ops/nextafter_native.h 2024-08-08T20:48:49.8208803Z inflating: build/lib/torch/include/ATen/ops/nextafter_ops.h 2024-08-08T20:48:49.8210001Z inflating: build/lib/torch/include/ATen/ops/nll_loss.h 2024-08-08T20:48:49.8211147Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d.h 2024-08-08T20:48:49.8212680Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_backward.h 2024-08-08T20:48:49.8213885Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.8215136Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.8216100Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_backward_native.h 2024-08-08T20:48:49.8217186Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_backward_ops.h 2024-08-08T20:48:49.8218444Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8219606Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_forward.h 2024-08-08T20:48:49.8220628Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_forward_cpu_dispatch.h 2024-08-08T20:48:49.8221911Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_forward_cuda_dispatch.h 2024-08-08T20:48:49.8222819Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_forward_native.h 2024-08-08T20:48:49.8223743Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_forward_ops.h 2024-08-08T20:48:49.8224535Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_native.h 2024-08-08T20:48:49.8225471Z inflating: build/lib/torch/include/ATen/ops/nll_loss2d_ops.h 2024-08-08T20:48:49.8226781Z inflating: build/lib/torch/include/ATen/ops/nll_loss_backward.h 2024-08-08T20:48:49.8228528Z inflating: build/lib/torch/include/ATen/ops/nll_loss_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8229621Z inflating: build/lib/torch/include/ATen/ops/nll_loss_backward_cpu_dispatch.h 2024-08-08T20:48:49.8230483Z inflating: build/lib/torch/include/ATen/ops/nll_loss_backward_cuda_dispatch.h 2024-08-08T20:48:49.8230999Z inflating: build/lib/torch/include/ATen/ops/nll_loss_backward_meta.h 2024-08-08T20:48:49.8231813Z inflating: build/lib/torch/include/ATen/ops/nll_loss_backward_meta_dispatch.h 2024-08-08T20:48:49.8233751Z inflating: build/lib/torch/include/ATen/ops/nll_loss_backward_native.h 2024-08-08T20:48:49.8234417Z inflating: build/lib/torch/include/ATen/ops/nll_loss_backward_ops.h 2024-08-08T20:48:49.8235719Z inflating: build/lib/torch/include/ATen/ops/nll_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8236820Z inflating: build/lib/torch/include/ATen/ops/nll_loss_forward.h 2024-08-08T20:48:49.8238273Z inflating: build/lib/torch/include/ATen/ops/nll_loss_forward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8239071Z inflating: build/lib/torch/include/ATen/ops/nll_loss_forward_cpu_dispatch.h 2024-08-08T20:48:49.8240331Z inflating: build/lib/torch/include/ATen/ops/nll_loss_forward_cuda_dispatch.h 2024-08-08T20:48:49.8241364Z inflating: build/lib/torch/include/ATen/ops/nll_loss_forward_meta.h 2024-08-08T20:48:49.8242477Z inflating: build/lib/torch/include/ATen/ops/nll_loss_forward_meta_dispatch.h 2024-08-08T20:48:49.8243383Z inflating: build/lib/torch/include/ATen/ops/nll_loss_forward_native.h 2024-08-08T20:48:49.8244697Z inflating: build/lib/torch/include/ATen/ops/nll_loss_forward_ops.h 2024-08-08T20:48:49.8245450Z inflating: build/lib/torch/include/ATen/ops/nll_loss_native.h 2024-08-08T20:48:49.8246421Z inflating: build/lib/torch/include/ATen/ops/nll_loss_nd.h 2024-08-08T20:48:49.8247881Z inflating: build/lib/torch/include/ATen/ops/nll_loss_nd_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8248489Z inflating: build/lib/torch/include/ATen/ops/nll_loss_nd_native.h 2024-08-08T20:48:49.8249245Z inflating: build/lib/torch/include/ATen/ops/nll_loss_nd_ops.h 2024-08-08T20:48:49.8250536Z inflating: build/lib/torch/include/ATen/ops/nll_loss_ops.h 2024-08-08T20:48:49.8251439Z inflating: build/lib/torch/include/ATen/ops/nonzero.h 2024-08-08T20:48:49.8252616Z inflating: build/lib/torch/include/ATen/ops/nonzero_cpu_dispatch.h 2024-08-08T20:48:49.8253639Z inflating: build/lib/torch/include/ATen/ops/nonzero_cuda_dispatch.h 2024-08-08T20:48:49.8254483Z inflating: build/lib/torch/include/ATen/ops/nonzero_native.h 2024-08-08T20:48:49.8255462Z inflating: build/lib/torch/include/ATen/ops/nonzero_numpy.h 2024-08-08T20:48:49.8256961Z inflating: build/lib/torch/include/ATen/ops/nonzero_numpy_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8257521Z inflating: build/lib/torch/include/ATen/ops/nonzero_numpy_native.h 2024-08-08T20:48:49.8258799Z inflating: build/lib/torch/include/ATen/ops/nonzero_numpy_ops.h 2024-08-08T20:48:49.8259687Z inflating: build/lib/torch/include/ATen/ops/nonzero_ops.h 2024-08-08T20:48:49.8260713Z inflating: build/lib/torch/include/ATen/ops/nonzero_static.h 2024-08-08T20:48:49.8261895Z inflating: build/lib/torch/include/ATen/ops/nonzero_static_cpu_dispatch.h 2024-08-08T20:48:49.8262777Z inflating: build/lib/torch/include/ATen/ops/nonzero_static_native.h 2024-08-08T20:48:49.8263771Z inflating: build/lib/torch/include/ATen/ops/nonzero_static_ops.h 2024-08-08T20:48:49.8264838Z inflating: build/lib/torch/include/ATen/ops/norm.h 2024-08-08T20:48:49.8266540Z inflating: build/lib/torch/include/ATen/ops/norm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8267779Z inflating: build/lib/torch/include/ATen/ops/norm_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8268900Z inflating: build/lib/torch/include/ATen/ops/norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8269522Z inflating: build/lib/torch/include/ATen/ops/norm_cpu_dispatch.h 2024-08-08T20:48:49.8270136Z inflating: build/lib/torch/include/ATen/ops/norm_cuda_dispatch.h 2024-08-08T20:48:49.8271097Z inflating: build/lib/torch/include/ATen/ops/norm_except_dim.h 2024-08-08T20:48:49.8272271Z inflating: build/lib/torch/include/ATen/ops/norm_except_dim_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8273136Z inflating: build/lib/torch/include/ATen/ops/norm_except_dim_native.h 2024-08-08T20:48:49.8274198Z inflating: build/lib/torch/include/ATen/ops/norm_except_dim_ops.h 2024-08-08T20:48:49.8275196Z inflating: build/lib/torch/include/ATen/ops/norm_meta.h 2024-08-08T20:48:49.8276519Z inflating: build/lib/torch/include/ATen/ops/norm_meta_dispatch.h 2024-08-08T20:48:49.8277523Z inflating: build/lib/torch/include/ATen/ops/norm_native.h 2024-08-08T20:48:49.8279247Z inflating: build/lib/torch/include/ATen/ops/norm_ops.h 2024-08-08T20:48:49.8280550Z inflating: build/lib/torch/include/ATen/ops/normal.h 2024-08-08T20:48:49.8281972Z inflating: build/lib/torch/include/ATen/ops/normal_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8282716Z inflating: build/lib/torch/include/ATen/ops/normal_cpu_dispatch.h 2024-08-08T20:48:49.8283832Z inflating: build/lib/torch/include/ATen/ops/normal_cuda_dispatch.h 2024-08-08T20:48:49.8284828Z inflating: build/lib/torch/include/ATen/ops/normal_meta_dispatch.h 2024-08-08T20:48:49.8285909Z inflating: build/lib/torch/include/ATen/ops/normal_native.h 2024-08-08T20:48:49.8287318Z inflating: build/lib/torch/include/ATen/ops/normal_ops.h 2024-08-08T20:48:49.8288521Z inflating: build/lib/torch/include/ATen/ops/not_equal.h 2024-08-08T20:48:49.8289987Z inflating: build/lib/torch/include/ATen/ops/not_equal_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8290564Z inflating: build/lib/torch/include/ATen/ops/not_equal_native.h 2024-08-08T20:48:49.8291676Z inflating: build/lib/torch/include/ATen/ops/not_equal_ops.h 2024-08-08T20:48:49.8292655Z inflating: build/lib/torch/include/ATen/ops/nuclear_norm.h 2024-08-08T20:48:49.8294146Z inflating: build/lib/torch/include/ATen/ops/nuclear_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8294780Z inflating: build/lib/torch/include/ATen/ops/nuclear_norm_native.h 2024-08-08T20:48:49.8295616Z inflating: build/lib/torch/include/ATen/ops/nuclear_norm_ops.h 2024-08-08T20:48:49.8296618Z inflating: build/lib/torch/include/ATen/ops/numpy_T.h 2024-08-08T20:48:49.8298109Z inflating: build/lib/torch/include/ATen/ops/numpy_T_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8298934Z inflating: build/lib/torch/include/ATen/ops/numpy_T_native.h 2024-08-08T20:48:49.8299515Z inflating: build/lib/torch/include/ATen/ops/numpy_T_ops.h 2024-08-08T20:48:49.8300497Z inflating: build/lib/torch/include/ATen/ops/one_hot.h 2024-08-08T20:48:49.8302891Z inflating: build/lib/torch/include/ATen/ops/one_hot_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8303469Z inflating: build/lib/torch/include/ATen/ops/one_hot_native.h 2024-08-08T20:48:49.8304171Z inflating: build/lib/torch/include/ATen/ops/one_hot_ops.h 2024-08-08T20:48:49.8305521Z inflating: build/lib/torch/include/ATen/ops/ones.h 2024-08-08T20:48:49.8306954Z inflating: build/lib/torch/include/ATen/ops/ones_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8307683Z inflating: build/lib/torch/include/ATen/ops/ones_like.h 2024-08-08T20:48:49.8309195Z inflating: build/lib/torch/include/ATen/ops/ones_like_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8309817Z inflating: build/lib/torch/include/ATen/ops/ones_like_native.h 2024-08-08T20:48:49.8310614Z inflating: build/lib/torch/include/ATen/ops/ones_like_ops.h 2024-08-08T20:48:49.8311456Z inflating: build/lib/torch/include/ATen/ops/ones_native.h 2024-08-08T20:48:49.8312930Z inflating: build/lib/torch/include/ATen/ops/ones_ops.h 2024-08-08T20:48:49.8313499Z inflating: build/lib/torch/include/ATen/ops/or.h 2024-08-08T20:48:49.8315037Z inflating: build/lib/torch/include/ATen/ops/or_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8315475Z inflating: build/lib/torch/include/ATen/ops/or_native.h 2024-08-08T20:48:49.8316908Z inflating: build/lib/torch/include/ATen/ops/or_ops.h 2024-08-08T20:48:49.8317461Z inflating: build/lib/torch/include/ATen/ops/orgqr.h 2024-08-08T20:48:49.8319144Z inflating: build/lib/torch/include/ATen/ops/orgqr_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8319791Z inflating: build/lib/torch/include/ATen/ops/orgqr_native.h 2024-08-08T20:48:49.8320655Z inflating: build/lib/torch/include/ATen/ops/orgqr_ops.h 2024-08-08T20:48:49.8321691Z inflating: build/lib/torch/include/ATen/ops/ormqr.h 2024-08-08T20:48:49.8323032Z inflating: build/lib/torch/include/ATen/ops/ormqr_cpu_dispatch.h 2024-08-08T20:48:49.8324057Z inflating: build/lib/torch/include/ATen/ops/ormqr_cuda_dispatch.h 2024-08-08T20:48:49.8324938Z inflating: build/lib/torch/include/ATen/ops/ormqr_native.h 2024-08-08T20:48:49.8325997Z inflating: build/lib/torch/include/ATen/ops/ormqr_ops.h 2024-08-08T20:48:49.8326865Z inflating: build/lib/torch/include/ATen/ops/outer.h 2024-08-08T20:48:49.8328295Z inflating: build/lib/torch/include/ATen/ops/outer_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8329029Z inflating: build/lib/torch/include/ATen/ops/outer_native.h 2024-08-08T20:48:49.8329915Z inflating: build/lib/torch/include/ATen/ops/outer_ops.h 2024-08-08T20:48:49.8330858Z inflating: build/lib/torch/include/ATen/ops/output_nr.h 2024-08-08T20:48:49.8332520Z inflating: build/lib/torch/include/ATen/ops/output_nr_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8333095Z inflating: build/lib/torch/include/ATen/ops/output_nr_native.h 2024-08-08T20:48:49.8334048Z inflating: build/lib/torch/include/ATen/ops/output_nr_ops.h 2024-08-08T20:48:49.8334982Z inflating: build/lib/torch/include/ATen/ops/pad.h 2024-08-08T20:48:49.8336457Z inflating: build/lib/torch/include/ATen/ops/pad_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8337049Z inflating: build/lib/torch/include/ATen/ops/pad_native.h 2024-08-08T20:48:49.8337742Z inflating: build/lib/torch/include/ATen/ops/pad_ops.h 2024-08-08T20:48:49.8338965Z inflating: build/lib/torch/include/ATen/ops/pad_sequence.h 2024-08-08T20:48:49.8340529Z inflating: build/lib/torch/include/ATen/ops/pad_sequence_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8341177Z inflating: build/lib/torch/include/ATen/ops/pad_sequence_native.h 2024-08-08T20:48:49.8341698Z inflating: build/lib/torch/include/ATen/ops/pad_sequence_ops.h 2024-08-08T20:48:49.8342724Z inflating: build/lib/torch/include/ATen/ops/pairwise_distance.h 2024-08-08T20:48:49.8343863Z inflating: build/lib/torch/include/ATen/ops/pairwise_distance_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8344994Z inflating: build/lib/torch/include/ATen/ops/pairwise_distance_native.h 2024-08-08T20:48:49.8346035Z inflating: build/lib/torch/include/ATen/ops/pairwise_distance_ops.h 2024-08-08T20:48:49.8346837Z inflating: build/lib/torch/include/ATen/ops/pdist.h 2024-08-08T20:48:49.8348331Z inflating: build/lib/torch/include/ATen/ops/pdist_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8348960Z inflating: build/lib/torch/include/ATen/ops/pdist_native.h 2024-08-08T20:48:49.8349494Z inflating: build/lib/torch/include/ATen/ops/pdist_ops.h 2024-08-08T20:48:49.8350596Z inflating: build/lib/torch/include/ATen/ops/permute.h 2024-08-08T20:48:49.8351778Z inflating: build/lib/torch/include/ATen/ops/permute_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8352574Z inflating: build/lib/torch/include/ATen/ops/permute_copy.h 2024-08-08T20:48:49.8353764Z inflating: build/lib/torch/include/ATen/ops/permute_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8354953Z inflating: build/lib/torch/include/ATen/ops/permute_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8355500Z inflating: build/lib/torch/include/ATen/ops/permute_copy_native.h 2024-08-08T20:48:49.8356560Z inflating: build/lib/torch/include/ATen/ops/permute_copy_ops.h 2024-08-08T20:48:49.8357434Z inflating: build/lib/torch/include/ATen/ops/permute_native.h 2024-08-08T20:48:49.8358852Z inflating: build/lib/torch/include/ATen/ops/permute_ops.h 2024-08-08T20:48:49.8359862Z inflating: build/lib/torch/include/ATen/ops/pin_memory.h 2024-08-08T20:48:49.8361619Z inflating: build/lib/torch/include/ATen/ops/pin_memory_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8362187Z inflating: build/lib/torch/include/ATen/ops/pin_memory_native.h 2024-08-08T20:48:49.8363107Z inflating: build/lib/torch/include/ATen/ops/pin_memory_ops.h 2024-08-08T20:48:49.8363940Z inflating: build/lib/torch/include/ATen/ops/pinverse.h 2024-08-08T20:48:49.8365287Z inflating: build/lib/torch/include/ATen/ops/pinverse_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8366049Z inflating: build/lib/torch/include/ATen/ops/pinverse_native.h 2024-08-08T20:48:49.8368091Z inflating: build/lib/torch/include/ATen/ops/pinverse_ops.h 2024-08-08T20:48:49.8368567Z inflating: build/lib/torch/include/ATen/ops/pixel_shuffle.h 2024-08-08T20:48:49.8370109Z inflating: build/lib/torch/include/ATen/ops/pixel_shuffle_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8371438Z inflating: build/lib/torch/include/ATen/ops/pixel_shuffle_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8372266Z inflating: build/lib/torch/include/ATen/ops/pixel_shuffle_cpu_dispatch.h 2024-08-08T20:48:49.8373128Z inflating: build/lib/torch/include/ATen/ops/pixel_shuffle_native.h 2024-08-08T20:48:49.8373939Z inflating: build/lib/torch/include/ATen/ops/pixel_shuffle_ops.h 2024-08-08T20:48:49.8374764Z inflating: build/lib/torch/include/ATen/ops/pixel_unshuffle.h 2024-08-08T20:48:49.8376254Z inflating: build/lib/torch/include/ATen/ops/pixel_unshuffle_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8377453Z inflating: build/lib/torch/include/ATen/ops/pixel_unshuffle_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8378307Z inflating: build/lib/torch/include/ATen/ops/pixel_unshuffle_cpu_dispatch.h 2024-08-08T20:48:49.8378957Z inflating: build/lib/torch/include/ATen/ops/pixel_unshuffle_native.h 2024-08-08T20:48:49.8379739Z inflating: build/lib/torch/include/ATen/ops/pixel_unshuffle_ops.h 2024-08-08T20:48:49.8380589Z inflating: build/lib/torch/include/ATen/ops/poisson.h 2024-08-08T20:48:49.8381718Z inflating: build/lib/torch/include/ATen/ops/poisson_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8382593Z inflating: build/lib/torch/include/ATen/ops/poisson_cpu_dispatch.h 2024-08-08T20:48:49.8383743Z inflating: build/lib/torch/include/ATen/ops/poisson_cuda_dispatch.h 2024-08-08T20:48:49.8384660Z inflating: build/lib/torch/include/ATen/ops/poisson_native.h 2024-08-08T20:48:49.8385623Z inflating: build/lib/torch/include/ATen/ops/poisson_nll_loss.h 2024-08-08T20:48:49.8387203Z inflating: build/lib/torch/include/ATen/ops/poisson_nll_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8387857Z inflating: build/lib/torch/include/ATen/ops/poisson_nll_loss_native.h 2024-08-08T20:48:49.8388654Z inflating: build/lib/torch/include/ATen/ops/poisson_nll_loss_ops.h 2024-08-08T20:48:49.8389704Z inflating: build/lib/torch/include/ATen/ops/poisson_ops.h 2024-08-08T20:48:49.8390532Z inflating: build/lib/torch/include/ATen/ops/polar.h 2024-08-08T20:48:49.8392017Z inflating: build/lib/torch/include/ATen/ops/polar_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8392853Z inflating: build/lib/torch/include/ATen/ops/polar_cpu_dispatch.h 2024-08-08T20:48:49.8393525Z inflating: build/lib/torch/include/ATen/ops/polar_cuda_dispatch.h 2024-08-08T20:48:49.8394265Z inflating: build/lib/torch/include/ATen/ops/polar_native.h 2024-08-08T20:48:49.8395282Z inflating: build/lib/torch/include/ATen/ops/polar_ops.h 2024-08-08T20:48:49.8396192Z inflating: build/lib/torch/include/ATen/ops/polygamma.h 2024-08-08T20:48:49.8397581Z inflating: build/lib/torch/include/ATen/ops/polygamma_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8398910Z inflating: build/lib/torch/include/ATen/ops/polygamma_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8399567Z inflating: build/lib/torch/include/ATen/ops/polygamma_cpu_dispatch.h 2024-08-08T20:48:49.8400648Z inflating: build/lib/torch/include/ATen/ops/polygamma_cuda_dispatch.h 2024-08-08T20:48:49.8401559Z inflating: build/lib/torch/include/ATen/ops/polygamma_meta.h 2024-08-08T20:48:49.8402958Z inflating: build/lib/torch/include/ATen/ops/polygamma_meta_dispatch.h 2024-08-08T20:48:49.8403788Z inflating: build/lib/torch/include/ATen/ops/polygamma_native.h 2024-08-08T20:48:49.8404873Z inflating: build/lib/torch/include/ATen/ops/polygamma_ops.h 2024-08-08T20:48:49.8405956Z inflating: build/lib/torch/include/ATen/ops/positive.h 2024-08-08T20:48:49.8407438Z inflating: build/lib/torch/include/ATen/ops/positive_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8408068Z inflating: build/lib/torch/include/ATen/ops/positive_native.h 2024-08-08T20:48:49.8408611Z inflating: build/lib/torch/include/ATen/ops/positive_ops.h 2024-08-08T20:48:49.8409674Z inflating: build/lib/torch/include/ATen/ops/pow.h 2024-08-08T20:48:49.8411158Z inflating: build/lib/torch/include/ATen/ops/pow_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8412000Z inflating: build/lib/torch/include/ATen/ops/pow_cpu_dispatch.h 2024-08-08T20:48:49.8412989Z inflating: build/lib/torch/include/ATen/ops/pow_cuda_dispatch.h 2024-08-08T20:48:49.8414084Z inflating: build/lib/torch/include/ATen/ops/pow_meta.h 2024-08-08T20:48:49.8415200Z inflating: build/lib/torch/include/ATen/ops/pow_meta_dispatch.h 2024-08-08T20:48:49.8416085Z inflating: build/lib/torch/include/ATen/ops/pow_native.h 2024-08-08T20:48:49.8417382Z inflating: build/lib/torch/include/ATen/ops/pow_ops.h 2024-08-08T20:48:49.8418240Z inflating: build/lib/torch/include/ATen/ops/prelu.h 2024-08-08T20:48:49.8419670Z inflating: build/lib/torch/include/ATen/ops/prelu_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8420411Z inflating: build/lib/torch/include/ATen/ops/prelu_native.h 2024-08-08T20:48:49.8421199Z inflating: build/lib/torch/include/ATen/ops/prelu_ops.h 2024-08-08T20:48:49.8422329Z inflating: build/lib/torch/include/ATen/ops/prod.h 2024-08-08T20:48:49.8423672Z inflating: build/lib/torch/include/ATen/ops/prod_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8424815Z inflating: build/lib/torch/include/ATen/ops/prod_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8425576Z inflating: build/lib/torch/include/ATen/ops/prod_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8426308Z inflating: build/lib/torch/include/ATen/ops/prod_cpu_dispatch.h 2024-08-08T20:48:49.8427565Z inflating: build/lib/torch/include/ATen/ops/prod_cuda_dispatch.h 2024-08-08T20:48:49.8428397Z inflating: build/lib/torch/include/ATen/ops/prod_meta.h 2024-08-08T20:48:49.8429496Z inflating: build/lib/torch/include/ATen/ops/prod_meta_dispatch.h 2024-08-08T20:48:49.8430388Z inflating: build/lib/torch/include/ATen/ops/prod_native.h 2024-08-08T20:48:49.8431681Z inflating: build/lib/torch/include/ATen/ops/prod_ops.h 2024-08-08T20:48:49.8433718Z inflating: build/lib/torch/include/ATen/ops/promote_types.h 2024-08-08T20:48:49.8434995Z inflating: build/lib/torch/include/ATen/ops/promote_types_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8435624Z inflating: build/lib/torch/include/ATen/ops/promote_types_native.h 2024-08-08T20:48:49.8436554Z inflating: build/lib/torch/include/ATen/ops/promote_types_ops.h 2024-08-08T20:48:49.8437384Z inflating: build/lib/torch/include/ATen/ops/put.h 2024-08-08T20:48:49.8438871Z inflating: build/lib/torch/include/ATen/ops/put_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8439614Z inflating: build/lib/torch/include/ATen/ops/put_cpu_dispatch.h 2024-08-08T20:48:49.8440241Z inflating: build/lib/torch/include/ATen/ops/put_cuda_dispatch.h 2024-08-08T20:48:49.8441250Z inflating: build/lib/torch/include/ATen/ops/put_meta_dispatch.h 2024-08-08T20:48:49.8442204Z inflating: build/lib/torch/include/ATen/ops/put_native.h 2024-08-08T20:48:49.8443566Z inflating: build/lib/torch/include/ATen/ops/put_ops.h 2024-08-08T20:48:49.8444571Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_axis.h 2024-08-08T20:48:49.8445561Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_axis_native.h 2024-08-08T20:48:49.8446487Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_axis_ops.h 2024-08-08T20:48:49.8447451Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_scales.h 2024-08-08T20:48:49.8448940Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_scales_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8449611Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_scales_native.h 2024-08-08T20:48:49.8450389Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_scales_ops.h 2024-08-08T20:48:49.8451642Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_zero_points.h 2024-08-08T20:48:49.8453176Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_zero_points_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8453948Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_zero_points_native.h 2024-08-08T20:48:49.8455120Z inflating: build/lib/torch/include/ATen/ops/q_per_channel_zero_points_ops.h 2024-08-08T20:48:49.8455621Z inflating: build/lib/torch/include/ATen/ops/q_scale.h 2024-08-08T20:48:49.8456369Z inflating: build/lib/torch/include/ATen/ops/q_scale_native.h 2024-08-08T20:48:49.8457681Z inflating: build/lib/torch/include/ATen/ops/q_zero_point.h 2024-08-08T20:48:49.8458912Z inflating: build/lib/torch/include/ATen/ops/q_zero_point_native.h 2024-08-08T20:48:49.8459892Z inflating: build/lib/torch/include/ATen/ops/q_zero_point_ops.h 2024-08-08T20:48:49.8460709Z inflating: build/lib/torch/include/ATen/ops/qr.h 2024-08-08T20:48:49.8462183Z inflating: build/lib/torch/include/ATen/ops/qr_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8462867Z inflating: build/lib/torch/include/ATen/ops/qr_native.h 2024-08-08T20:48:49.8463673Z inflating: build/lib/torch/include/ATen/ops/qr_ops.h 2024-08-08T20:48:49.8464860Z inflating: build/lib/torch/include/ATen/ops/qscheme.h 2024-08-08T20:48:49.8465847Z inflating: build/lib/torch/include/ATen/ops/qscheme_native.h 2024-08-08T20:48:49.8466902Z inflating: build/lib/torch/include/ATen/ops/qscheme_ops.h 2024-08-08T20:48:49.8467906Z inflating: build/lib/torch/include/ATen/ops/quantile.h 2024-08-08T20:48:49.8469381Z inflating: build/lib/torch/include/ATen/ops/quantile_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8470066Z inflating: build/lib/torch/include/ATen/ops/quantile_native.h 2024-08-08T20:48:49.8470872Z inflating: build/lib/torch/include/ATen/ops/quantile_ops.h 2024-08-08T20:48:49.8471828Z inflating: build/lib/torch/include/ATen/ops/quantize_per_channel.h 2024-08-08T20:48:49.8473257Z inflating: build/lib/torch/include/ATen/ops/quantize_per_channel_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8474127Z inflating: build/lib/torch/include/ATen/ops/quantize_per_channel_cpu_dispatch.h 2024-08-08T20:48:49.8475301Z inflating: build/lib/torch/include/ATen/ops/quantize_per_channel_cuda_dispatch.h 2024-08-08T20:48:49.8476328Z inflating: build/lib/torch/include/ATen/ops/quantize_per_channel_native.h 2024-08-08T20:48:49.8477378Z inflating: build/lib/torch/include/ATen/ops/quantize_per_channel_ops.h 2024-08-08T20:48:49.8478446Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor.h 2024-08-08T20:48:49.8480020Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8480881Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_cpu_dispatch.h 2024-08-08T20:48:49.8481592Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_cuda_dispatch.h 2024-08-08T20:48:49.8482534Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_dynamic.h 2024-08-08T20:48:49.8484213Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_dynamic_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8485248Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_dynamic_cpu_dispatch.h 2024-08-08T20:48:49.8486048Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_dynamic_cuda_dispatch.h 2024-08-08T20:48:49.8486859Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_dynamic_native.h 2024-08-08T20:48:49.8487603Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_dynamic_ops.h 2024-08-08T20:48:49.8488090Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_native.h 2024-08-08T20:48:49.8489669Z inflating: build/lib/torch/include/ATen/ops/quantize_per_tensor_ops.h 2024-08-08T20:48:49.8490388Z inflating: build/lib/torch/include/ATen/ops/quantized_batch_norm.h 2024-08-08T20:48:49.8491747Z inflating: build/lib/torch/include/ATen/ops/quantized_batch_norm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8492502Z inflating: build/lib/torch/include/ATen/ops/quantized_batch_norm_native.h 2024-08-08T20:48:49.8493602Z inflating: build/lib/torch/include/ATen/ops/quantized_batch_norm_ops.h 2024-08-08T20:48:49.8494653Z inflating: build/lib/torch/include/ATen/ops/quantized_gru_cell.h 2024-08-08T20:48:49.8495928Z inflating: build/lib/torch/include/ATen/ops/quantized_gru_cell_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8496529Z inflating: build/lib/torch/include/ATen/ops/quantized_gru_cell_native.h 2024-08-08T20:48:49.8498154Z inflating: build/lib/torch/include/ATen/ops/quantized_gru_cell_ops.h 2024-08-08T20:48:49.8499702Z inflating: build/lib/torch/include/ATen/ops/quantized_lstm_cell.h 2024-08-08T20:48:49.8500629Z inflating: build/lib/torch/include/ATen/ops/quantized_lstm_cell_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8501480Z inflating: build/lib/torch/include/ATen/ops/quantized_lstm_cell_native.h 2024-08-08T20:48:49.8502568Z inflating: build/lib/torch/include/ATen/ops/quantized_lstm_cell_ops.h 2024-08-08T20:48:49.8503664Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool1d.h 2024-08-08T20:48:49.8504789Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool1d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8505614Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool1d_native.h 2024-08-08T20:48:49.8506687Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool1d_ops.h 2024-08-08T20:48:49.8507587Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool2d.h 2024-08-08T20:48:49.8508983Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool2d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8509636Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool2d_native.h 2024-08-08T20:48:49.8510741Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool2d_ops.h 2024-08-08T20:48:49.8511805Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool3d.h 2024-08-08T20:48:49.8513023Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool3d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8513864Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool3d_native.h 2024-08-08T20:48:49.8514924Z inflating: build/lib/torch/include/ATen/ops/quantized_max_pool3d_ops.h 2024-08-08T20:48:49.8515851Z inflating: build/lib/torch/include/ATen/ops/quantized_rnn_relu_cell.h 2024-08-08T20:48:49.8517255Z inflating: build/lib/torch/include/ATen/ops/quantized_rnn_relu_cell_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8518022Z inflating: build/lib/torch/include/ATen/ops/quantized_rnn_relu_cell_native.h 2024-08-08T20:48:49.8519080Z inflating: build/lib/torch/include/ATen/ops/quantized_rnn_relu_cell_ops.h 2024-08-08T20:48:49.8519993Z inflating: build/lib/torch/include/ATen/ops/quantized_rnn_tanh_cell.h 2024-08-08T20:48:49.8521453Z inflating: build/lib/torch/include/ATen/ops/quantized_rnn_tanh_cell_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8522511Z inflating: build/lib/torch/include/ATen/ops/quantized_rnn_tanh_cell_native.h 2024-08-08T20:48:49.8523222Z inflating: build/lib/torch/include/ATen/ops/quantized_rnn_tanh_cell_ops.h 2024-08-08T20:48:49.8523851Z inflating: build/lib/torch/include/ATen/ops/rad2deg.h 2024-08-08T20:48:49.8525119Z inflating: build/lib/torch/include/ATen/ops/rad2deg_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8525819Z inflating: build/lib/torch/include/ATen/ops/rad2deg_native.h 2024-08-08T20:48:49.8526892Z inflating: build/lib/torch/include/ATen/ops/rad2deg_ops.h 2024-08-08T20:48:49.8529309Z inflating: build/lib/torch/include/ATen/ops/rand.h 2024-08-08T20:48:49.8530479Z inflating: build/lib/torch/include/ATen/ops/rand_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8531491Z inflating: build/lib/torch/include/ATen/ops/rand_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8532523Z inflating: build/lib/torch/include/ATen/ops/rand_like.h 2024-08-08T20:48:49.8533856Z inflating: build/lib/torch/include/ATen/ops/rand_like_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8534698Z inflating: build/lib/torch/include/ATen/ops/rand_like_native.h 2024-08-08T20:48:49.8535684Z inflating: build/lib/torch/include/ATen/ops/rand_like_ops.h 2024-08-08T20:48:49.8536629Z inflating: build/lib/torch/include/ATen/ops/rand_native.h 2024-08-08T20:48:49.8538401Z inflating: build/lib/torch/include/ATen/ops/rand_ops.h 2024-08-08T20:48:49.8540519Z inflating: build/lib/torch/include/ATen/ops/randint.h 2024-08-08T20:48:49.8541706Z inflating: build/lib/torch/include/ATen/ops/randint_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8543450Z inflating: build/lib/torch/include/ATen/ops/randint_like.h 2024-08-08T20:48:49.8544542Z inflating: build/lib/torch/include/ATen/ops/randint_like_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8545306Z inflating: build/lib/torch/include/ATen/ops/randint_like_native.h 2024-08-08T20:48:49.8546878Z inflating: build/lib/torch/include/ATen/ops/randint_like_ops.h 2024-08-08T20:48:49.8547359Z inflating: build/lib/torch/include/ATen/ops/randint_native.h 2024-08-08T20:48:49.8549128Z inflating: build/lib/torch/include/ATen/ops/randint_ops.h 2024-08-08T20:48:49.8551183Z inflating: build/lib/torch/include/ATen/ops/randn.h 2024-08-08T20:48:49.8552359Z inflating: build/lib/torch/include/ATen/ops/randn_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8553368Z inflating: build/lib/torch/include/ATen/ops/randn_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8554169Z inflating: build/lib/torch/include/ATen/ops/randn_like.h 2024-08-08T20:48:49.8555527Z inflating: build/lib/torch/include/ATen/ops/randn_like_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8556609Z inflating: build/lib/torch/include/ATen/ops/randn_like_compositeimplicitautogradnestedtensor_dispatch.h 2024-08-08T20:48:49.8557231Z inflating: build/lib/torch/include/ATen/ops/randn_like_native.h 2024-08-08T20:48:49.8558311Z inflating: build/lib/torch/include/ATen/ops/randn_like_ops.h 2024-08-08T20:48:49.8559568Z inflating: build/lib/torch/include/ATen/ops/randn_native.h 2024-08-08T20:48:49.8561246Z inflating: build/lib/torch/include/ATen/ops/randn_ops.h 2024-08-08T20:48:49.8561905Z inflating: build/lib/torch/include/ATen/ops/random.h 2024-08-08T20:48:49.8563591Z inflating: build/lib/torch/include/ATen/ops/random_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8564215Z inflating: build/lib/torch/include/ATen/ops/random_cpu_dispatch.h 2024-08-08T20:48:49.8565242Z inflating: build/lib/torch/include/ATen/ops/random_cuda_dispatch.h 2024-08-08T20:48:49.8566160Z inflating: build/lib/torch/include/ATen/ops/random_meta_dispatch.h 2024-08-08T20:48:49.8567106Z inflating: build/lib/torch/include/ATen/ops/random_native.h 2024-08-08T20:48:49.8568796Z inflating: build/lib/torch/include/ATen/ops/random_ops.h 2024-08-08T20:48:49.8569933Z inflating: build/lib/torch/include/ATen/ops/randperm.h 2024-08-08T20:48:49.8571768Z inflating: build/lib/torch/include/ATen/ops/randperm_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8572752Z inflating: build/lib/torch/include/ATen/ops/randperm_cpu_dispatch.h 2024-08-08T20:48:49.8573730Z inflating: build/lib/torch/include/ATen/ops/randperm_cuda_dispatch.h 2024-08-08T20:48:49.8574670Z inflating: build/lib/torch/include/ATen/ops/randperm_native.h 2024-08-08T20:48:49.8576292Z inflating: build/lib/torch/include/ATen/ops/randperm_ops.h 2024-08-08T20:48:49.8576870Z inflating: build/lib/torch/include/ATen/ops/range.h 2024-08-08T20:48:49.8578227Z inflating: build/lib/torch/include/ATen/ops/range_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8579010Z inflating: build/lib/torch/include/ATen/ops/range_cpu_dispatch.h 2024-08-08T20:48:49.8579991Z inflating: build/lib/torch/include/ATen/ops/range_cuda_dispatch.h 2024-08-08T20:48:49.8580917Z inflating: build/lib/torch/include/ATen/ops/range_meta_dispatch.h 2024-08-08T20:48:49.8581860Z inflating: build/lib/torch/include/ATen/ops/range_native.h 2024-08-08T20:48:49.8583774Z inflating: build/lib/torch/include/ATen/ops/range_ops.h 2024-08-08T20:48:49.8584251Z inflating: build/lib/torch/include/ATen/ops/ravel.h 2024-08-08T20:48:49.8585436Z inflating: build/lib/torch/include/ATen/ops/ravel_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8586117Z inflating: build/lib/torch/include/ATen/ops/ravel_native.h 2024-08-08T20:48:49.8587159Z inflating: build/lib/torch/include/ATen/ops/ravel_ops.h 2024-08-08T20:48:49.8588050Z inflating: build/lib/torch/include/ATen/ops/real.h 2024-08-08T20:48:49.8589355Z inflating: build/lib/torch/include/ATen/ops/real_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8590047Z inflating: build/lib/torch/include/ATen/ops/real_native.h 2024-08-08T20:48:49.8591003Z inflating: build/lib/torch/include/ATen/ops/real_ops.h 2024-08-08T20:48:49.8592000Z inflating: build/lib/torch/include/ATen/ops/reciprocal.h 2024-08-08T20:48:49.8593434Z inflating: build/lib/torch/include/ATen/ops/reciprocal_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8594075Z inflating: build/lib/torch/include/ATen/ops/reciprocal_cpu_dispatch.h 2024-08-08T20:48:49.8595034Z inflating: build/lib/torch/include/ATen/ops/reciprocal_cuda_dispatch.h 2024-08-08T20:48:49.8595908Z inflating: build/lib/torch/include/ATen/ops/reciprocal_meta.h 2024-08-08T20:48:49.8596921Z inflating: build/lib/torch/include/ATen/ops/reciprocal_meta_dispatch.h 2024-08-08T20:48:49.8597849Z inflating: build/lib/torch/include/ATen/ops/reciprocal_native.h 2024-08-08T20:48:49.8598914Z inflating: build/lib/torch/include/ATen/ops/reciprocal_ops.h 2024-08-08T20:48:49.8599813Z inflating: build/lib/torch/include/ATen/ops/record_stream.h 2024-08-08T20:48:49.8600902Z inflating: build/lib/torch/include/ATen/ops/record_stream_cuda_dispatch.h 2024-08-08T20:48:49.8601810Z inflating: build/lib/torch/include/ATen/ops/record_stream_native.h 2024-08-08T20:48:49.8602793Z inflating: build/lib/torch/include/ATen/ops/record_stream_ops.h 2024-08-08T20:48:49.8603902Z inflating: build/lib/torch/include/ATen/ops/refine_names.h 2024-08-08T20:48:49.8605211Z inflating: build/lib/torch/include/ATen/ops/refine_names_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8605995Z inflating: build/lib/torch/include/ATen/ops/refine_names_native.h 2024-08-08T20:48:49.8606924Z inflating: build/lib/torch/include/ATen/ops/refine_names_ops.h 2024-08-08T20:48:49.8608034Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d.h 2024-08-08T20:48:49.8609442Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_backward.h 2024-08-08T20:48:49.8610726Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8611478Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_backward_cpu_dispatch.h 2024-08-08T20:48:49.8612532Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_backward_cuda_dispatch.h 2024-08-08T20:48:49.8613379Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_backward_meta.h 2024-08-08T20:48:49.8614535Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_backward_meta_dispatch.h 2024-08-08T20:48:49.8615423Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_backward_native.h 2024-08-08T20:48:49.8616477Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_backward_ops.h 2024-08-08T20:48:49.8617752Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8618496Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_cpu_dispatch.h 2024-08-08T20:48:49.8619518Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_cuda_dispatch.h 2024-08-08T20:48:49.8620398Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_meta.h 2024-08-08T20:48:49.8621475Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_meta_dispatch.h 2024-08-08T20:48:49.8622474Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_native.h 2024-08-08T20:48:49.8623440Z inflating: build/lib/torch/include/ATen/ops/reflection_pad1d_ops.h 2024-08-08T20:48:49.8624476Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d.h 2024-08-08T20:48:49.8626033Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d_backward.h 2024-08-08T20:48:49.8626735Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.8627879Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.8628794Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d_backward_native.h 2024-08-08T20:48:49.8629858Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d_backward_ops.h 2024-08-08T20:48:49.8630881Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d_cpu_dispatch.h 2024-08-08T20:48:49.8631936Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d_cuda_dispatch.h 2024-08-08T20:48:49.8632876Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d_native.h 2024-08-08T20:48:49.8633893Z inflating: build/lib/torch/include/ATen/ops/reflection_pad2d_ops.h 2024-08-08T20:48:49.8634984Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d.h 2024-08-08T20:48:49.8636516Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_backward.h 2024-08-08T20:48:49.8637886Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8639164Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_backward_cpu_dispatch.h 2024-08-08T20:48:49.8640851Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_backward_cuda_dispatch.h 2024-08-08T20:48:49.8641608Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_backward_meta.h 2024-08-08T20:48:49.8642716Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_backward_meta_dispatch.h 2024-08-08T20:48:49.8643430Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_backward_native.h 2024-08-08T20:48:49.8644554Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_backward_ops.h 2024-08-08T20:48:49.8646056Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8646777Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_cpu_dispatch.h 2024-08-08T20:48:49.8647757Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_cuda_dispatch.h 2024-08-08T20:48:49.8648494Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_meta.h 2024-08-08T20:48:49.8649416Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_meta_dispatch.h 2024-08-08T20:48:49.8650539Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_native.h 2024-08-08T20:48:49.8652028Z inflating: build/lib/torch/include/ATen/ops/reflection_pad3d_ops.h 2024-08-08T20:48:49.8652488Z inflating: build/lib/torch/include/ATen/ops/relu.h 2024-08-08T20:48:49.8653246Z inflating: build/lib/torch/include/ATen/ops/relu6.h 2024-08-08T20:48:49.8654702Z inflating: build/lib/torch/include/ATen/ops/relu6_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8655284Z inflating: build/lib/torch/include/ATen/ops/relu6_native.h 2024-08-08T20:48:49.8655866Z inflating: build/lib/torch/include/ATen/ops/relu6_ops.h 2024-08-08T20:48:49.8657359Z inflating: build/lib/torch/include/ATen/ops/relu_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8658138Z inflating: build/lib/torch/include/ATen/ops/relu_cpu_dispatch.h 2024-08-08T20:48:49.8659407Z inflating: build/lib/torch/include/ATen/ops/relu_cuda_dispatch.h 2024-08-08T20:48:49.8660499Z inflating: build/lib/torch/include/ATen/ops/relu_meta_dispatch.h 2024-08-08T20:48:49.8661403Z inflating: build/lib/torch/include/ATen/ops/relu_native.h 2024-08-08T20:48:49.8662632Z inflating: build/lib/torch/include/ATen/ops/relu_ops.h 2024-08-08T20:48:49.8663632Z inflating: build/lib/torch/include/ATen/ops/remainder.h 2024-08-08T20:48:49.8665033Z inflating: build/lib/torch/include/ATen/ops/remainder_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8666103Z inflating: build/lib/torch/include/ATen/ops/remainder_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8666691Z inflating: build/lib/torch/include/ATen/ops/remainder_cpu_dispatch.h 2024-08-08T20:48:49.8667378Z inflating: build/lib/torch/include/ATen/ops/remainder_cuda_dispatch.h 2024-08-08T20:48:49.8668386Z inflating: build/lib/torch/include/ATen/ops/remainder_meta.h 2024-08-08T20:48:49.8669345Z inflating: build/lib/torch/include/ATen/ops/remainder_meta_dispatch.h 2024-08-08T20:48:49.8670508Z inflating: build/lib/torch/include/ATen/ops/remainder_native.h 2024-08-08T20:48:49.8672094Z inflating: build/lib/torch/include/ATen/ops/remainder_ops.h 2024-08-08T20:48:49.8672697Z inflating: build/lib/torch/include/ATen/ops/rename.h 2024-08-08T20:48:49.8674445Z inflating: build/lib/torch/include/ATen/ops/rename_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8675027Z inflating: build/lib/torch/include/ATen/ops/rename_native.h 2024-08-08T20:48:49.8675607Z inflating: build/lib/torch/include/ATen/ops/rename_ops.h 2024-08-08T20:48:49.8676822Z inflating: build/lib/torch/include/ATen/ops/renorm.h 2024-08-08T20:48:49.8678046Z inflating: build/lib/torch/include/ATen/ops/renorm_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8678844Z inflating: build/lib/torch/include/ATen/ops/renorm_cpu_dispatch.h 2024-08-08T20:48:49.8679753Z inflating: build/lib/torch/include/ATen/ops/renorm_cuda_dispatch.h 2024-08-08T20:48:49.8680716Z inflating: build/lib/torch/include/ATen/ops/renorm_meta.h 2024-08-08T20:48:49.8681683Z inflating: build/lib/torch/include/ATen/ops/renorm_meta_dispatch.h 2024-08-08T20:48:49.8682644Z inflating: build/lib/torch/include/ATen/ops/renorm_native.h 2024-08-08T20:48:49.8683724Z inflating: build/lib/torch/include/ATen/ops/renorm_ops.h 2024-08-08T20:48:49.8684775Z inflating: build/lib/torch/include/ATen/ops/repeat.h 2024-08-08T20:48:49.8686096Z inflating: build/lib/torch/include/ATen/ops/repeat_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8687182Z inflating: build/lib/torch/include/ATen/ops/repeat_interleave.h 2024-08-08T20:48:49.8688591Z inflating: build/lib/torch/include/ATen/ops/repeat_interleave_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8689704Z inflating: build/lib/torch/include/ATen/ops/repeat_interleave_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8690421Z inflating: build/lib/torch/include/ATen/ops/repeat_interleave_cpu_dispatch.h 2024-08-08T20:48:49.8691569Z inflating: build/lib/torch/include/ATen/ops/repeat_interleave_cuda_dispatch.h 2024-08-08T20:48:49.8692656Z inflating: build/lib/torch/include/ATen/ops/repeat_interleave_native.h 2024-08-08T20:48:49.8693972Z inflating: build/lib/torch/include/ATen/ops/repeat_interleave_ops.h 2024-08-08T20:48:49.8694492Z inflating: build/lib/torch/include/ATen/ops/repeat_native.h 2024-08-08T20:48:49.8704712Z inflating: build/lib/torch/include/ATen/ops/repeat_ops.h 2024-08-08T20:48:49.8705791Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d.h 2024-08-08T20:48:49.8706614Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_backward.h 2024-08-08T20:48:49.8707916Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8708742Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_backward_cpu_dispatch.h 2024-08-08T20:48:49.8709590Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_backward_cuda_dispatch.h 2024-08-08T20:48:49.8710394Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_backward_meta.h 2024-08-08T20:48:49.8711369Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_backward_meta_dispatch.h 2024-08-08T20:48:49.8712173Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_backward_native.h 2024-08-08T20:48:49.8712822Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_backward_ops.h 2024-08-08T20:48:49.8713956Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8714922Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_cpu_dispatch.h 2024-08-08T20:48:49.8715630Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_cuda_dispatch.h 2024-08-08T20:48:49.8716310Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_meta.h 2024-08-08T20:48:49.8717051Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_meta_dispatch.h 2024-08-08T20:48:49.8717711Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_native.h 2024-08-08T20:48:49.8718375Z inflating: build/lib/torch/include/ATen/ops/replication_pad1d_ops.h 2024-08-08T20:48:49.8718934Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d.h 2024-08-08T20:48:49.8719632Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_backward.h 2024-08-08T20:48:49.8720504Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_backward_cpu_dispatch.h 2024-08-08T20:48:49.8721483Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_backward_cuda_dispatch.h 2024-08-08T20:48:49.8722201Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_backward_native.h 2024-08-08T20:48:49.8722960Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_backward_ops.h 2024-08-08T20:48:49.8724115Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8724848Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_cpu_dispatch.h 2024-08-08T20:48:49.8725578Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_cuda_dispatch.h 2024-08-08T20:48:49.8726341Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_meta.h 2024-08-08T20:48:49.8727056Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_meta_dispatch.h 2024-08-08T20:48:49.8727689Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_native.h 2024-08-08T20:48:49.8728285Z inflating: build/lib/torch/include/ATen/ops/replication_pad2d_ops.h 2024-08-08T20:48:49.8728881Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d.h 2024-08-08T20:48:49.8729668Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_backward.h 2024-08-08T20:48:49.8730531Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_backward_cpu_dispatch.h 2024-08-08T20:48:49.8731714Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_backward_cuda_dispatch.h 2024-08-08T20:48:49.8732487Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_backward_native.h 2024-08-08T20:48:49.8733317Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_backward_ops.h 2024-08-08T20:48:49.8734509Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8735329Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_cpu_dispatch.h 2024-08-08T20:48:49.8736062Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_cuda_dispatch.h 2024-08-08T20:48:49.8736839Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_meta.h 2024-08-08T20:48:49.8737727Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_meta_dispatch.h 2024-08-08T20:48:49.8738409Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_native.h 2024-08-08T20:48:49.8739385Z inflating: build/lib/torch/include/ATen/ops/replication_pad3d_ops.h 2024-08-08T20:48:49.8740132Z inflating: build/lib/torch/include/ATen/ops/requires_grad.h 2024-08-08T20:48:49.8741734Z inflating: build/lib/torch/include/ATen/ops/requires_grad_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8742436Z inflating: build/lib/torch/include/ATen/ops/requires_grad_native.h 2024-08-08T20:48:49.8742989Z inflating: build/lib/torch/include/ATen/ops/requires_grad_ops.h 2024-08-08T20:48:49.8743555Z inflating: build/lib/torch/include/ATen/ops/reshape.h 2024-08-08T20:48:49.8744630Z inflating: build/lib/torch/include/ATen/ops/reshape_as.h 2024-08-08T20:48:49.8745838Z inflating: build/lib/torch/include/ATen/ops/reshape_as_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8747043Z inflating: build/lib/torch/include/ATen/ops/reshape_as_compositeimplicitautogradnestedtensor_dispatch.h 2024-08-08T20:48:49.8747586Z inflating: build/lib/torch/include/ATen/ops/reshape_as_native.h 2024-08-08T20:48:49.8748661Z inflating: build/lib/torch/include/ATen/ops/reshape_as_ops.h 2024-08-08T20:48:49.8749858Z inflating: build/lib/torch/include/ATen/ops/reshape_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8751018Z inflating: build/lib/torch/include/ATen/ops/reshape_compositeimplicitautogradnestedtensor_dispatch.h 2024-08-08T20:48:49.8751581Z inflating: build/lib/torch/include/ATen/ops/reshape_native.h 2024-08-08T20:48:49.8752654Z inflating: build/lib/torch/include/ATen/ops/reshape_ops.h 2024-08-08T20:48:49.8754315Z inflating: build/lib/torch/include/ATen/ops/resize.h 2024-08-08T20:48:49.8754932Z inflating: build/lib/torch/include/ATen/ops/resize_as.h 2024-08-08T20:48:49.8756571Z inflating: build/lib/torch/include/ATen/ops/resize_as_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8757059Z inflating: build/lib/torch/include/ATen/ops/resize_as_native.h 2024-08-08T20:48:49.8757990Z inflating: build/lib/torch/include/ATen/ops/resize_as_ops.h 2024-08-08T20:48:49.8759635Z inflating: build/lib/torch/include/ATen/ops/resize_as_sparse.h 2024-08-08T20:48:49.8760897Z inflating: build/lib/torch/include/ATen/ops/resize_as_sparse_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8761791Z inflating: build/lib/torch/include/ATen/ops/resize_as_sparse_meta_dispatch.h 2024-08-08T20:48:49.8762678Z inflating: build/lib/torch/include/ATen/ops/resize_as_sparse_native.h 2024-08-08T20:48:49.8763760Z inflating: build/lib/torch/include/ATen/ops/resize_as_sparse_ops.h 2024-08-08T20:48:49.8765187Z inflating: build/lib/torch/include/ATen/ops/resize_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8766019Z inflating: build/lib/torch/include/ATen/ops/resize_cpu_dispatch.h 2024-08-08T20:48:49.8766881Z inflating: build/lib/torch/include/ATen/ops/resize_cuda_dispatch.h 2024-08-08T20:48:49.8767896Z inflating: build/lib/torch/include/ATen/ops/resize_meta_dispatch.h 2024-08-08T20:48:49.8768849Z inflating: build/lib/torch/include/ATen/ops/resize_native.h 2024-08-08T20:48:49.8770014Z inflating: build/lib/torch/include/ATen/ops/resize_ops.h 2024-08-08T20:48:49.8771946Z inflating: build/lib/torch/include/ATen/ops/resolve_conj.h 2024-08-08T20:48:49.8772877Z inflating: build/lib/torch/include/ATen/ops/resolve_conj_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8773542Z inflating: build/lib/torch/include/ATen/ops/resolve_conj_native.h 2024-08-08T20:48:49.8774305Z inflating: build/lib/torch/include/ATen/ops/resolve_conj_ops.h 2024-08-08T20:48:49.8775311Z inflating: build/lib/torch/include/ATen/ops/resolve_neg.h 2024-08-08T20:48:49.8776568Z inflating: build/lib/torch/include/ATen/ops/resolve_neg_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8777339Z inflating: build/lib/torch/include/ATen/ops/resolve_neg_native.h 2024-08-08T20:48:49.8778239Z inflating: build/lib/torch/include/ATen/ops/resolve_neg_ops.h 2024-08-08T20:48:49.8779262Z inflating: build/lib/torch/include/ATen/ops/result_type.h 2024-08-08T20:48:49.8781157Z inflating: build/lib/torch/include/ATen/ops/result_type_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8782103Z inflating: build/lib/torch/include/ATen/ops/result_type_native.h 2024-08-08T20:48:49.8783086Z inflating: build/lib/torch/include/ATen/ops/result_type_ops.h 2024-08-08T20:48:49.8783554Z inflating: build/lib/torch/include/ATen/ops/retain_grad.h 2024-08-08T20:48:49.8784887Z inflating: build/lib/torch/include/ATen/ops/retain_grad_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8785510Z inflating: build/lib/torch/include/ATen/ops/retain_grad_native.h 2024-08-08T20:48:49.8786570Z inflating: build/lib/torch/include/ATen/ops/retain_grad_ops.h 2024-08-08T20:48:49.8787579Z inflating: build/lib/torch/include/ATen/ops/retains_grad.h 2024-08-08T20:48:49.8789324Z inflating: build/lib/torch/include/ATen/ops/retains_grad_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8790008Z inflating: build/lib/torch/include/ATen/ops/retains_grad_native.h 2024-08-08T20:48:49.8790813Z inflating: build/lib/torch/include/ATen/ops/retains_grad_ops.h 2024-08-08T20:48:49.8791712Z inflating: build/lib/torch/include/ATen/ops/rms_norm.h 2024-08-08T20:48:49.8793203Z inflating: build/lib/torch/include/ATen/ops/rms_norm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8793712Z inflating: build/lib/torch/include/ATen/ops/rms_norm_native.h 2024-08-08T20:48:49.8794635Z inflating: build/lib/torch/include/ATen/ops/rms_norm_ops.h 2024-08-08T20:48:49.8795805Z inflating: build/lib/torch/include/ATen/ops/rnn_relu.h 2024-08-08T20:48:49.8796823Z inflating: build/lib/torch/include/ATen/ops/rnn_relu_cell.h 2024-08-08T20:48:49.8798409Z inflating: build/lib/torch/include/ATen/ops/rnn_relu_cell_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8798907Z inflating: build/lib/torch/include/ATen/ops/rnn_relu_cell_native.h 2024-08-08T20:48:49.8799604Z inflating: build/lib/torch/include/ATen/ops/rnn_relu_cell_ops.h 2024-08-08T20:48:49.8801332Z inflating: build/lib/torch/include/ATen/ops/rnn_relu_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8801952Z inflating: build/lib/torch/include/ATen/ops/rnn_relu_native.h 2024-08-08T20:48:49.8803284Z inflating: build/lib/torch/include/ATen/ops/rnn_relu_ops.h 2024-08-08T20:48:49.8804346Z inflating: build/lib/torch/include/ATen/ops/rnn_tanh.h 2024-08-08T20:48:49.8805392Z inflating: build/lib/torch/include/ATen/ops/rnn_tanh_cell.h 2024-08-08T20:48:49.8806847Z inflating: build/lib/torch/include/ATen/ops/rnn_tanh_cell_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8807374Z inflating: build/lib/torch/include/ATen/ops/rnn_tanh_cell_native.h 2024-08-08T20:48:49.8808052Z inflating: build/lib/torch/include/ATen/ops/rnn_tanh_cell_ops.h 2024-08-08T20:48:49.8809416Z inflating: build/lib/torch/include/ATen/ops/rnn_tanh_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8810038Z inflating: build/lib/torch/include/ATen/ops/rnn_tanh_native.h 2024-08-08T20:48:49.8811269Z inflating: build/lib/torch/include/ATen/ops/rnn_tanh_ops.h 2024-08-08T20:48:49.8812567Z inflating: build/lib/torch/include/ATen/ops/roll.h 2024-08-08T20:48:49.8813620Z inflating: build/lib/torch/include/ATen/ops/roll_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8814520Z inflating: build/lib/torch/include/ATen/ops/roll_cpu_dispatch.h 2024-08-08T20:48:49.8815416Z inflating: build/lib/torch/include/ATen/ops/roll_cuda_dispatch.h 2024-08-08T20:48:49.8816421Z inflating: build/lib/torch/include/ATen/ops/roll_native.h 2024-08-08T20:48:49.8817384Z inflating: build/lib/torch/include/ATen/ops/roll_ops.h 2024-08-08T20:48:49.8818412Z inflating: build/lib/torch/include/ATen/ops/rot90.h 2024-08-08T20:48:49.8819752Z inflating: build/lib/torch/include/ATen/ops/rot90_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8820354Z inflating: build/lib/torch/include/ATen/ops/rot90_native.h 2024-08-08T20:48:49.8821596Z inflating: build/lib/torch/include/ATen/ops/rot90_ops.h 2024-08-08T20:48:49.8822430Z inflating: build/lib/torch/include/ATen/ops/round.h 2024-08-08T20:48:49.8824375Z inflating: build/lib/torch/include/ATen/ops/round_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8824784Z inflating: build/lib/torch/include/ATen/ops/round_cpu_dispatch.h 2024-08-08T20:48:49.8825520Z inflating: build/lib/torch/include/ATen/ops/round_cuda_dispatch.h 2024-08-08T20:48:49.8826568Z inflating: build/lib/torch/include/ATen/ops/round_meta.h 2024-08-08T20:48:49.8827534Z inflating: build/lib/torch/include/ATen/ops/round_meta_dispatch.h 2024-08-08T20:48:49.8828562Z inflating: build/lib/torch/include/ATen/ops/round_native.h 2024-08-08T20:48:49.8829927Z inflating: build/lib/torch/include/ATen/ops/round_ops.h 2024-08-08T20:48:49.8830575Z inflating: build/lib/torch/include/ATen/ops/row_indices.h 2024-08-08T20:48:49.8831958Z inflating: build/lib/torch/include/ATen/ops/row_indices_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8832800Z inflating: build/lib/torch/include/ATen/ops/row_indices_copy.h 2024-08-08T20:48:49.8834621Z inflating: build/lib/torch/include/ATen/ops/row_indices_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8835814Z inflating: build/lib/torch/include/ATen/ops/row_indices_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8836295Z inflating: build/lib/torch/include/ATen/ops/row_indices_copy_native.h 2024-08-08T20:48:49.8837818Z inflating: build/lib/torch/include/ATen/ops/row_indices_copy_ops.h 2024-08-08T20:48:49.8838727Z inflating: build/lib/torch/include/ATen/ops/row_indices_native.h 2024-08-08T20:48:49.8839759Z inflating: build/lib/torch/include/ATen/ops/row_indices_ops.h 2024-08-08T20:48:49.8840734Z inflating: build/lib/torch/include/ATen/ops/row_stack.h 2024-08-08T20:48:49.8842208Z inflating: build/lib/torch/include/ATen/ops/row_stack_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8842910Z inflating: build/lib/torch/include/ATen/ops/row_stack_native.h 2024-08-08T20:48:49.8843772Z inflating: build/lib/torch/include/ATen/ops/row_stack_ops.h 2024-08-08T20:48:49.8844774Z inflating: build/lib/torch/include/ATen/ops/rrelu.h 2024-08-08T20:48:49.8846144Z inflating: build/lib/torch/include/ATen/ops/rrelu_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8846888Z inflating: build/lib/torch/include/ATen/ops/rrelu_native.h 2024-08-08T20:48:49.8847903Z inflating: build/lib/torch/include/ATen/ops/rrelu_ops.h 2024-08-08T20:48:49.8849016Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise.h 2024-08-08T20:48:49.8850169Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise_backward.h 2024-08-08T20:48:49.8851666Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8852384Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise_backward_native.h 2024-08-08T20:48:49.8853449Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise_backward_ops.h 2024-08-08T20:48:49.8854527Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise_cpu_dispatch.h 2024-08-08T20:48:49.8855568Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise_cuda_dispatch.h 2024-08-08T20:48:49.8856571Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise_meta_dispatch.h 2024-08-08T20:48:49.8857557Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise_native.h 2024-08-08T20:48:49.8859053Z inflating: build/lib/torch/include/ATen/ops/rrelu_with_noise_ops.h 2024-08-08T20:48:49.8860214Z inflating: build/lib/torch/include/ATen/ops/rshift.h 2024-08-08T20:48:49.8861618Z inflating: build/lib/torch/include/ATen/ops/rshift_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8862466Z inflating: build/lib/torch/include/ATen/ops/rshift_cpu_dispatch.h 2024-08-08T20:48:49.8863332Z inflating: build/lib/torch/include/ATen/ops/rshift_cuda_dispatch.h 2024-08-08T20:48:49.8864314Z inflating: build/lib/torch/include/ATen/ops/rshift_meta_dispatch.h 2024-08-08T20:48:49.8865375Z inflating: build/lib/torch/include/ATen/ops/rshift_native.h 2024-08-08T20:48:49.8866532Z inflating: build/lib/torch/include/ATen/ops/rshift_ops.h 2024-08-08T20:48:49.8867325Z inflating: build/lib/torch/include/ATen/ops/rsqrt.h 2024-08-08T20:48:49.8869107Z inflating: build/lib/torch/include/ATen/ops/rsqrt_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8869688Z inflating: build/lib/torch/include/ATen/ops/rsqrt_cpu_dispatch.h 2024-08-08T20:48:49.8870705Z inflating: build/lib/torch/include/ATen/ops/rsqrt_cuda_dispatch.h 2024-08-08T20:48:49.8871528Z inflating: build/lib/torch/include/ATen/ops/rsqrt_meta.h 2024-08-08T20:48:49.8872570Z inflating: build/lib/torch/include/ATen/ops/rsqrt_meta_dispatch.h 2024-08-08T20:48:49.8873488Z inflating: build/lib/torch/include/ATen/ops/rsqrt_native.h 2024-08-08T20:48:49.8874533Z inflating: build/lib/torch/include/ATen/ops/rsqrt_ops.h 2024-08-08T20:48:49.8875498Z inflating: build/lib/torch/include/ATen/ops/rsub.h 2024-08-08T20:48:49.8876896Z inflating: build/lib/torch/include/ATen/ops/rsub_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8877705Z inflating: build/lib/torch/include/ATen/ops/rsub_cpu_dispatch.h 2024-08-08T20:48:49.8878548Z inflating: build/lib/torch/include/ATen/ops/rsub_cuda_dispatch.h 2024-08-08T20:48:49.8879429Z inflating: build/lib/torch/include/ATen/ops/rsub_native.h 2024-08-08T20:48:49.8880584Z inflating: build/lib/torch/include/ATen/ops/rsub_ops.h 2024-08-08T20:48:49.8881648Z inflating: build/lib/torch/include/ATen/ops/scalar_tensor.h 2024-08-08T20:48:49.8883152Z inflating: build/lib/torch/include/ATen/ops/scalar_tensor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8883786Z inflating: build/lib/torch/include/ATen/ops/scalar_tensor_native.h 2024-08-08T20:48:49.8884704Z inflating: build/lib/torch/include/ATen/ops/scalar_tensor_ops.h 2024-08-08T20:48:49.8885817Z inflating: build/lib/torch/include/ATen/ops/scaled_dot_product_attention.h 2024-08-08T20:48:49.8887358Z inflating: build/lib/torch/include/ATen/ops/scaled_dot_product_attention_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8888165Z inflating: build/lib/torch/include/ATen/ops/scaled_dot_product_attention_native.h 2024-08-08T20:48:49.8889012Z inflating: build/lib/torch/include/ATen/ops/scaled_dot_product_attention_ops.h 2024-08-08T20:48:49.8889653Z inflating: build/lib/torch/include/ATen/ops/scatter.h 2024-08-08T20:48:49.8890649Z inflating: build/lib/torch/include/ATen/ops/scatter_add.h 2024-08-08T20:48:49.8892640Z inflating: build/lib/torch/include/ATen/ops/scatter_add_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8893705Z inflating: build/lib/torch/include/ATen/ops/scatter_add_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8894420Z inflating: build/lib/torch/include/ATen/ops/scatter_add_cpu_dispatch.h 2024-08-08T20:48:49.8895486Z inflating: build/lib/torch/include/ATen/ops/scatter_add_cuda_dispatch.h 2024-08-08T20:48:49.8896021Z inflating: build/lib/torch/include/ATen/ops/scatter_add_meta.h 2024-08-08T20:48:49.8896933Z inflating: build/lib/torch/include/ATen/ops/scatter_add_meta_dispatch.h 2024-08-08T20:48:49.8898059Z inflating: build/lib/torch/include/ATen/ops/scatter_add_native.h 2024-08-08T20:48:49.8899180Z inflating: build/lib/torch/include/ATen/ops/scatter_add_ops.h 2024-08-08T20:48:49.8900783Z inflating: build/lib/torch/include/ATen/ops/scatter_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8901690Z inflating: build/lib/torch/include/ATen/ops/scatter_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8902499Z inflating: build/lib/torch/include/ATen/ops/scatter_cpu_dispatch.h 2024-08-08T20:48:49.8903525Z inflating: build/lib/torch/include/ATen/ops/scatter_cuda_dispatch.h 2024-08-08T20:48:49.8905287Z inflating: build/lib/torch/include/ATen/ops/scatter_meta.h 2024-08-08T20:48:49.8906305Z inflating: build/lib/torch/include/ATen/ops/scatter_meta_dispatch.h 2024-08-08T20:48:49.8907356Z inflating: build/lib/torch/include/ATen/ops/scatter_native.h 2024-08-08T20:48:49.8909111Z inflating: build/lib/torch/include/ATen/ops/scatter_ops.h 2024-08-08T20:48:49.8909624Z inflating: build/lib/torch/include/ATen/ops/scatter_reduce.h 2024-08-08T20:48:49.8911071Z inflating: build/lib/torch/include/ATen/ops/scatter_reduce_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8911901Z inflating: build/lib/torch/include/ATen/ops/scatter_reduce_cpu_dispatch.h 2024-08-08T20:48:49.8913147Z inflating: build/lib/torch/include/ATen/ops/scatter_reduce_cuda_dispatch.h 2024-08-08T20:48:49.8913897Z inflating: build/lib/torch/include/ATen/ops/scatter_reduce_meta.h 2024-08-08T20:48:49.8915063Z inflating: build/lib/torch/include/ATen/ops/scatter_reduce_meta_dispatch.h 2024-08-08T20:48:49.8915874Z inflating: build/lib/torch/include/ATen/ops/scatter_reduce_native.h 2024-08-08T20:48:49.8916980Z inflating: build/lib/torch/include/ATen/ops/scatter_reduce_ops.h 2024-08-08T20:48:49.8918100Z inflating: build/lib/torch/include/ATen/ops/searchsorted.h 2024-08-08T20:48:49.8919264Z inflating: build/lib/torch/include/ATen/ops/searchsorted_cpu_dispatch.h 2024-08-08T20:48:49.8920277Z inflating: build/lib/torch/include/ATen/ops/searchsorted_cuda_dispatch.h 2024-08-08T20:48:49.8921279Z inflating: build/lib/torch/include/ATen/ops/searchsorted_native.h 2024-08-08T20:48:49.8922489Z inflating: build/lib/torch/include/ATen/ops/searchsorted_ops.h 2024-08-08T20:48:49.8923537Z inflating: build/lib/torch/include/ATen/ops/segment_reduce.h 2024-08-08T20:48:49.8924915Z inflating: build/lib/torch/include/ATen/ops/segment_reduce_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8925807Z inflating: build/lib/torch/include/ATen/ops/segment_reduce_cpu_dispatch.h 2024-08-08T20:48:49.8926715Z inflating: build/lib/torch/include/ATen/ops/segment_reduce_cuda_dispatch.h 2024-08-08T20:48:49.8927632Z inflating: build/lib/torch/include/ATen/ops/segment_reduce_native.h 2024-08-08T20:48:49.8928773Z inflating: build/lib/torch/include/ATen/ops/segment_reduce_ops.h 2024-08-08T20:48:49.8929629Z inflating: build/lib/torch/include/ATen/ops/select.h 2024-08-08T20:48:49.8930895Z inflating: build/lib/torch/include/ATen/ops/select_backward.h 2024-08-08T20:48:49.8932406Z inflating: build/lib/torch/include/ATen/ops/select_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8933355Z inflating: build/lib/torch/include/ATen/ops/select_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8934042Z inflating: build/lib/torch/include/ATen/ops/select_backward_native.h 2024-08-08T20:48:49.8935134Z inflating: build/lib/torch/include/ATen/ops/select_backward_ops.h 2024-08-08T20:48:49.8936406Z inflating: build/lib/torch/include/ATen/ops/select_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8937355Z inflating: build/lib/torch/include/ATen/ops/select_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8938219Z inflating: build/lib/torch/include/ATen/ops/select_copy.h 2024-08-08T20:48:49.8939586Z inflating: build/lib/torch/include/ATen/ops/select_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8940608Z inflating: build/lib/torch/include/ATen/ops/select_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8941294Z inflating: build/lib/torch/include/ATen/ops/select_copy_native.h 2024-08-08T20:48:49.8942438Z inflating: build/lib/torch/include/ATen/ops/select_copy_ops.h 2024-08-08T20:48:49.8943572Z inflating: build/lib/torch/include/ATen/ops/select_native.h 2024-08-08T20:48:49.8944536Z inflating: build/lib/torch/include/ATen/ops/select_ops.h 2024-08-08T20:48:49.8945753Z inflating: build/lib/torch/include/ATen/ops/select_scatter.h 2024-08-08T20:48:49.8947202Z inflating: build/lib/torch/include/ATen/ops/select_scatter_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8948314Z inflating: build/lib/torch/include/ATen/ops/select_scatter_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8948753Z inflating: build/lib/torch/include/ATen/ops/select_scatter_native.h 2024-08-08T20:48:49.8949929Z inflating: build/lib/torch/include/ATen/ops/select_scatter_ops.h 2024-08-08T20:48:49.8950321Z inflating: build/lib/torch/include/ATen/ops/selu.h 2024-08-08T20:48:49.8951647Z inflating: build/lib/torch/include/ATen/ops/selu_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8952456Z inflating: build/lib/torch/include/ATen/ops/selu_native.h 2024-08-08T20:48:49.8953659Z inflating: build/lib/torch/include/ATen/ops/selu_ops.h 2024-08-08T20:48:49.8954997Z inflating: build/lib/torch/include/ATen/ops/set.h 2024-08-08T20:48:49.8956520Z inflating: build/lib/torch/include/ATen/ops/set_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.8957550Z inflating: build/lib/torch/include/ATen/ops/set_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8958334Z inflating: build/lib/torch/include/ATen/ops/set_cpu_dispatch.h 2024-08-08T20:48:49.8959473Z inflating: build/lib/torch/include/ATen/ops/set_cuda_dispatch.h 2024-08-08T20:48:49.8960360Z inflating: build/lib/torch/include/ATen/ops/set_data.h 2024-08-08T20:48:49.8961886Z inflating: build/lib/torch/include/ATen/ops/set_data_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.8962418Z inflating: build/lib/torch/include/ATen/ops/set_data_native.h 2024-08-08T20:48:49.8963397Z inflating: build/lib/torch/include/ATen/ops/set_data_ops.h 2024-08-08T20:48:49.8964409Z inflating: build/lib/torch/include/ATen/ops/set_meta_dispatch.h 2024-08-08T20:48:49.8965371Z inflating: build/lib/torch/include/ATen/ops/set_native.h 2024-08-08T20:48:49.8966875Z inflating: build/lib/torch/include/ATen/ops/set_ops.h 2024-08-08T20:48:49.8967717Z inflating: build/lib/torch/include/ATen/ops/sgn.h 2024-08-08T20:48:49.8969285Z inflating: build/lib/torch/include/ATen/ops/sgn_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8969869Z inflating: build/lib/torch/include/ATen/ops/sgn_cpu_dispatch.h 2024-08-08T20:48:49.8970503Z inflating: build/lib/torch/include/ATen/ops/sgn_cuda_dispatch.h 2024-08-08T20:48:49.8971591Z inflating: build/lib/torch/include/ATen/ops/sgn_meta.h 2024-08-08T20:48:49.8973473Z inflating: build/lib/torch/include/ATen/ops/sgn_meta_dispatch.h 2024-08-08T20:48:49.8974009Z inflating: build/lib/torch/include/ATen/ops/sgn_native.h 2024-08-08T20:48:49.8975271Z inflating: build/lib/torch/include/ATen/ops/sgn_ops.h 2024-08-08T20:48:49.8976269Z inflating: build/lib/torch/include/ATen/ops/sigmoid.h 2024-08-08T20:48:49.8977529Z inflating: build/lib/torch/include/ATen/ops/sigmoid_backward.h 2024-08-08T20:48:49.8979311Z inflating: build/lib/torch/include/ATen/ops/sigmoid_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8980442Z inflating: build/lib/torch/include/ATen/ops/sigmoid_backward_cpu_dispatch.h 2024-08-08T20:48:49.8981151Z inflating: build/lib/torch/include/ATen/ops/sigmoid_backward_cuda_dispatch.h 2024-08-08T20:48:49.8982048Z inflating: build/lib/torch/include/ATen/ops/sigmoid_backward_meta.h 2024-08-08T20:48:49.8982974Z inflating: build/lib/torch/include/ATen/ops/sigmoid_backward_meta_dispatch.h 2024-08-08T20:48:49.8983781Z inflating: build/lib/torch/include/ATen/ops/sigmoid_backward_native.h 2024-08-08T20:48:49.8984865Z inflating: build/lib/torch/include/ATen/ops/sigmoid_backward_ops.h 2024-08-08T20:48:49.8986375Z inflating: build/lib/torch/include/ATen/ops/sigmoid_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8987025Z inflating: build/lib/torch/include/ATen/ops/sigmoid_cpu_dispatch.h 2024-08-08T20:48:49.8987867Z inflating: build/lib/torch/include/ATen/ops/sigmoid_cuda_dispatch.h 2024-08-08T20:48:49.8988673Z inflating: build/lib/torch/include/ATen/ops/sigmoid_meta.h 2024-08-08T20:48:49.8989782Z inflating: build/lib/torch/include/ATen/ops/sigmoid_meta_dispatch.h 2024-08-08T20:48:49.8990871Z inflating: build/lib/torch/include/ATen/ops/sigmoid_native.h 2024-08-08T20:48:49.8991781Z inflating: build/lib/torch/include/ATen/ops/sigmoid_ops.h 2024-08-08T20:48:49.8992595Z inflating: build/lib/torch/include/ATen/ops/sign.h 2024-08-08T20:48:49.8994333Z inflating: build/lib/torch/include/ATen/ops/sign_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.8994949Z inflating: build/lib/torch/include/ATen/ops/sign_cpu_dispatch.h 2024-08-08T20:48:49.8995519Z inflating: build/lib/torch/include/ATen/ops/sign_cuda_dispatch.h 2024-08-08T20:48:49.8996336Z inflating: build/lib/torch/include/ATen/ops/sign_meta.h 2024-08-08T20:48:49.8997267Z inflating: build/lib/torch/include/ATen/ops/sign_meta_dispatch.h 2024-08-08T20:48:49.8998304Z inflating: build/lib/torch/include/ATen/ops/sign_native.h 2024-08-08T20:48:49.8999301Z inflating: build/lib/torch/include/ATen/ops/sign_ops.h 2024-08-08T20:48:49.9000339Z inflating: build/lib/torch/include/ATen/ops/signbit.h 2024-08-08T20:48:49.9001628Z inflating: build/lib/torch/include/ATen/ops/signbit_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9002444Z inflating: build/lib/torch/include/ATen/ops/signbit_cpu_dispatch.h 2024-08-08T20:48:49.9003314Z inflating: build/lib/torch/include/ATen/ops/signbit_cuda_dispatch.h 2024-08-08T20:48:49.9004438Z inflating: build/lib/torch/include/ATen/ops/signbit_meta.h 2024-08-08T20:48:49.9005407Z inflating: build/lib/torch/include/ATen/ops/signbit_meta_dispatch.h 2024-08-08T20:48:49.9006416Z inflating: build/lib/torch/include/ATen/ops/signbit_native.h 2024-08-08T20:48:49.9007328Z inflating: build/lib/torch/include/ATen/ops/signbit_ops.h 2024-08-08T20:48:49.9008357Z inflating: build/lib/torch/include/ATen/ops/silu.h 2024-08-08T20:48:49.9009616Z inflating: build/lib/torch/include/ATen/ops/silu_backward.h 2024-08-08T20:48:49.9011441Z inflating: build/lib/torch/include/ATen/ops/silu_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9012575Z inflating: build/lib/torch/include/ATen/ops/silu_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9013381Z inflating: build/lib/torch/include/ATen/ops/silu_backward_cpu_dispatch.h 2024-08-08T20:48:49.9014199Z inflating: build/lib/torch/include/ATen/ops/silu_backward_cuda_dispatch.h 2024-08-08T20:48:49.9014784Z inflating: build/lib/torch/include/ATen/ops/silu_backward_meta.h 2024-08-08T20:48:49.9015401Z inflating: build/lib/torch/include/ATen/ops/silu_backward_meta_dispatch.h 2024-08-08T20:48:49.9016321Z inflating: build/lib/torch/include/ATen/ops/silu_backward_native.h 2024-08-08T20:48:49.9017414Z inflating: build/lib/torch/include/ATen/ops/silu_backward_ops.h 2024-08-08T20:48:49.9019156Z inflating: build/lib/torch/include/ATen/ops/silu_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9020045Z inflating: build/lib/torch/include/ATen/ops/silu_cpu_dispatch.h 2024-08-08T20:48:49.9020544Z inflating: build/lib/torch/include/ATen/ops/silu_cuda_dispatch.h 2024-08-08T20:48:49.9021266Z inflating: build/lib/torch/include/ATen/ops/silu_meta.h 2024-08-08T20:48:49.9022592Z inflating: build/lib/torch/include/ATen/ops/silu_meta_dispatch.h 2024-08-08T20:48:49.9023549Z inflating: build/lib/torch/include/ATen/ops/silu_native.h 2024-08-08T20:48:49.9024617Z inflating: build/lib/torch/include/ATen/ops/silu_ops.h 2024-08-08T20:48:49.9025486Z inflating: build/lib/torch/include/ATen/ops/sin.h 2024-08-08T20:48:49.9027156Z inflating: build/lib/torch/include/ATen/ops/sin_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9027780Z inflating: build/lib/torch/include/ATen/ops/sin_cpu_dispatch.h 2024-08-08T20:48:49.9028539Z inflating: build/lib/torch/include/ATen/ops/sin_cuda_dispatch.h 2024-08-08T20:48:49.9029411Z inflating: build/lib/torch/include/ATen/ops/sin_meta.h 2024-08-08T20:48:49.9030482Z inflating: build/lib/torch/include/ATen/ops/sin_meta_dispatch.h 2024-08-08T20:48:49.9031512Z inflating: build/lib/torch/include/ATen/ops/sin_native.h 2024-08-08T20:48:49.9032536Z inflating: build/lib/torch/include/ATen/ops/sin_ops.h 2024-08-08T20:48:49.9033314Z inflating: build/lib/torch/include/ATen/ops/sinc.h 2024-08-08T20:48:49.9035095Z inflating: build/lib/torch/include/ATen/ops/sinc_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9035680Z inflating: build/lib/torch/include/ATen/ops/sinc_cpu_dispatch.h 2024-08-08T20:48:49.9036567Z inflating: build/lib/torch/include/ATen/ops/sinc_cuda_dispatch.h 2024-08-08T20:48:49.9038094Z inflating: build/lib/torch/include/ATen/ops/sinc_meta.h 2024-08-08T20:48:49.9039193Z inflating: build/lib/torch/include/ATen/ops/sinc_meta_dispatch.h 2024-08-08T20:48:49.9040043Z inflating: build/lib/torch/include/ATen/ops/sinc_native.h 2024-08-08T20:48:49.9041136Z inflating: build/lib/torch/include/ATen/ops/sinc_ops.h 2024-08-08T20:48:49.9042051Z inflating: build/lib/torch/include/ATen/ops/sinh.h 2024-08-08T20:48:49.9043795Z inflating: build/lib/torch/include/ATen/ops/sinh_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9044431Z inflating: build/lib/torch/include/ATen/ops/sinh_cpu_dispatch.h 2024-08-08T20:48:49.9045265Z inflating: build/lib/torch/include/ATen/ops/sinh_cuda_dispatch.h 2024-08-08T20:48:49.9045991Z inflating: build/lib/torch/include/ATen/ops/sinh_meta.h 2024-08-08T20:48:49.9047056Z inflating: build/lib/torch/include/ATen/ops/sinh_meta_dispatch.h 2024-08-08T20:48:49.9047942Z inflating: build/lib/torch/include/ATen/ops/sinh_native.h 2024-08-08T20:48:49.9048956Z inflating: build/lib/torch/include/ATen/ops/sinh_ops.h 2024-08-08T20:48:49.9049871Z inflating: build/lib/torch/include/ATen/ops/size.h 2024-08-08T20:48:49.9051255Z inflating: build/lib/torch/include/ATen/ops/size_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9051892Z inflating: build/lib/torch/include/ATen/ops/size_native.h 2024-08-08T20:48:49.9052989Z inflating: build/lib/torch/include/ATen/ops/size_ops.h 2024-08-08T20:48:49.9053976Z inflating: build/lib/torch/include/ATen/ops/slice.h 2024-08-08T20:48:49.9055340Z inflating: build/lib/torch/include/ATen/ops/slice_backward.h 2024-08-08T20:48:49.9056944Z inflating: build/lib/torch/include/ATen/ops/slice_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9057561Z inflating: build/lib/torch/include/ATen/ops/slice_backward_native.h 2024-08-08T20:48:49.9058822Z inflating: build/lib/torch/include/ATen/ops/slice_backward_ops.h 2024-08-08T20:48:49.9059999Z inflating: build/lib/torch/include/ATen/ops/slice_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9061025Z inflating: build/lib/torch/include/ATen/ops/slice_copy.h 2024-08-08T20:48:49.9062429Z inflating: build/lib/torch/include/ATen/ops/slice_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9063527Z inflating: build/lib/torch/include/ATen/ops/slice_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9064269Z inflating: build/lib/torch/include/ATen/ops/slice_copy_native.h 2024-08-08T20:48:49.9065298Z inflating: build/lib/torch/include/ATen/ops/slice_copy_ops.h 2024-08-08T20:48:49.9066307Z inflating: build/lib/torch/include/ATen/ops/slice_inverse.h 2024-08-08T20:48:49.9067848Z inflating: build/lib/torch/include/ATen/ops/slice_inverse_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9068675Z inflating: build/lib/torch/include/ATen/ops/slice_inverse_native.h 2024-08-08T20:48:49.9069419Z inflating: build/lib/torch/include/ATen/ops/slice_inverse_ops.h 2024-08-08T20:48:49.9070596Z inflating: build/lib/torch/include/ATen/ops/slice_native.h 2024-08-08T20:48:49.9071600Z inflating: build/lib/torch/include/ATen/ops/slice_ops.h 2024-08-08T20:48:49.9072920Z inflating: build/lib/torch/include/ATen/ops/slice_scatter.h 2024-08-08T20:48:49.9074339Z inflating: build/lib/torch/include/ATen/ops/slice_scatter_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9076026Z inflating: build/lib/torch/include/ATen/ops/slice_scatter_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9076848Z inflating: build/lib/torch/include/ATen/ops/slice_scatter_native.h 2024-08-08T20:48:49.9077416Z inflating: build/lib/torch/include/ATen/ops/slice_scatter_ops.h 2024-08-08T20:48:49.9077901Z inflating: build/lib/torch/include/ATen/ops/slogdet.h 2024-08-08T20:48:49.9079215Z inflating: build/lib/torch/include/ATen/ops/slogdet_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9080012Z inflating: build/lib/torch/include/ATen/ops/slogdet_native.h 2024-08-08T20:48:49.9081021Z inflating: build/lib/torch/include/ATen/ops/slogdet_ops.h 2024-08-08T20:48:49.9082314Z inflating: build/lib/torch/include/ATen/ops/slow_conv3d.h 2024-08-08T20:48:49.9083692Z inflating: build/lib/torch/include/ATen/ops/slow_conv3d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9084834Z inflating: build/lib/torch/include/ATen/ops/slow_conv3d_forward.h 2024-08-08T20:48:49.9085996Z inflating: build/lib/torch/include/ATen/ops/slow_conv3d_forward_cpu_dispatch.h 2024-08-08T20:48:49.9086831Z inflating: build/lib/torch/include/ATen/ops/slow_conv3d_forward_native.h 2024-08-08T20:48:49.9087930Z inflating: build/lib/torch/include/ATen/ops/slow_conv3d_forward_ops.h 2024-08-08T20:48:49.9088836Z inflating: build/lib/torch/include/ATen/ops/slow_conv3d_native.h 2024-08-08T20:48:49.9089970Z inflating: build/lib/torch/include/ATen/ops/slow_conv3d_ops.h 2024-08-08T20:48:49.9091266Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated2d.h 2024-08-08T20:48:49.9092791Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated2d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9093368Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated2d_cpu_dispatch.h 2024-08-08T20:48:49.9094117Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated2d_cuda_dispatch.h 2024-08-08T20:48:49.9095152Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated2d_native.h 2024-08-08T20:48:49.9096226Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated2d_ops.h 2024-08-08T20:48:49.9097967Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated3d.h 2024-08-08T20:48:49.9098714Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated3d_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9099672Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated3d_cpu_dispatch.h 2024-08-08T20:48:49.9100587Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated3d_cuda_dispatch.h 2024-08-08T20:48:49.9101600Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated3d_native.h 2024-08-08T20:48:49.9102717Z inflating: build/lib/torch/include/ATen/ops/slow_conv_dilated3d_ops.h 2024-08-08T20:48:49.9104130Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose2d.h 2024-08-08T20:48:49.9105766Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose2d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9107423Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose2d_cpu_dispatch.h 2024-08-08T20:48:49.9108230Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose2d_cuda_dispatch.h 2024-08-08T20:48:49.9109252Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose2d_meta.h 2024-08-08T20:48:49.9110471Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose2d_meta_dispatch.h 2024-08-08T20:48:49.9111666Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose2d_native.h 2024-08-08T20:48:49.9112886Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose2d_ops.h 2024-08-08T20:48:49.9114346Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose3d.h 2024-08-08T20:48:49.9115505Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose3d_cpu_dispatch.h 2024-08-08T20:48:49.9116859Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose3d_cuda_dispatch.h 2024-08-08T20:48:49.9117648Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose3d_native.h 2024-08-08T20:48:49.9118729Z inflating: build/lib/torch/include/ATen/ops/slow_conv_transpose3d_ops.h 2024-08-08T20:48:49.9119421Z inflating: build/lib/torch/include/ATen/ops/smm.h 2024-08-08T20:48:49.9120819Z inflating: build/lib/torch/include/ATen/ops/smm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9121544Z inflating: build/lib/torch/include/ATen/ops/smm_native.h 2024-08-08T20:48:49.9122414Z inflating: build/lib/torch/include/ATen/ops/smm_ops.h 2024-08-08T20:48:49.9123534Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss.h 2024-08-08T20:48:49.9124655Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_backward.h 2024-08-08T20:48:49.9126176Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9127148Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_backward_cpu_dispatch.h 2024-08-08T20:48:49.9128076Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_backward_cuda_dispatch.h 2024-08-08T20:48:49.9128817Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_backward_native.h 2024-08-08T20:48:49.9129745Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_backward_ops.h 2024-08-08T20:48:49.9131366Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9132203Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_cpu_dispatch.h 2024-08-08T20:48:49.9132959Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_cuda_dispatch.h 2024-08-08T20:48:49.9133740Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_meta.h 2024-08-08T20:48:49.9134838Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_meta_dispatch.h 2024-08-08T20:48:49.9135679Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_native.h 2024-08-08T20:48:49.9136778Z inflating: build/lib/torch/include/ATen/ops/smooth_l1_loss_ops.h 2024-08-08T20:48:49.9137784Z inflating: build/lib/torch/include/ATen/ops/soft_margin_loss.h 2024-08-08T20:48:49.9138827Z inflating: build/lib/torch/include/ATen/ops/soft_margin_loss_backward.h 2024-08-08T20:48:49.9140623Z inflating: build/lib/torch/include/ATen/ops/soft_margin_loss_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9141382Z inflating: build/lib/torch/include/ATen/ops/soft_margin_loss_backward_native.h 2024-08-08T20:48:49.9142355Z inflating: build/lib/torch/include/ATen/ops/soft_margin_loss_backward_ops.h 2024-08-08T20:48:49.9143642Z inflating: build/lib/torch/include/ATen/ops/soft_margin_loss_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9144514Z inflating: build/lib/torch/include/ATen/ops/soft_margin_loss_native.h 2024-08-08T20:48:49.9145163Z inflating: build/lib/torch/include/ATen/ops/soft_margin_loss_ops.h 2024-08-08T20:48:49.9145693Z inflating: build/lib/torch/include/ATen/ops/softmax.h 2024-08-08T20:48:49.9147103Z inflating: build/lib/torch/include/ATen/ops/softmax_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9148026Z inflating: build/lib/torch/include/ATen/ops/softmax_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9148908Z inflating: build/lib/torch/include/ATen/ops/softmax_native.h 2024-08-08T20:48:49.9150000Z inflating: build/lib/torch/include/ATen/ops/softmax_ops.h 2024-08-08T20:48:49.9151372Z inflating: build/lib/torch/include/ATen/ops/softplus.h 2024-08-08T20:48:49.9151980Z inflating: build/lib/torch/include/ATen/ops/softplus_backward.h 2024-08-08T20:48:49.9153483Z inflating: build/lib/torch/include/ATen/ops/softplus_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9154124Z inflating: build/lib/torch/include/ATen/ops/softplus_backward_cpu_dispatch.h 2024-08-08T20:48:49.9155384Z inflating: build/lib/torch/include/ATen/ops/softplus_backward_cuda_dispatch.h 2024-08-08T20:48:49.9156057Z inflating: build/lib/torch/include/ATen/ops/softplus_backward_meta.h 2024-08-08T20:48:49.9157259Z inflating: build/lib/torch/include/ATen/ops/softplus_backward_meta_dispatch.h 2024-08-08T20:48:49.9158679Z inflating: build/lib/torch/include/ATen/ops/softplus_backward_native.h 2024-08-08T20:48:49.9159867Z inflating: build/lib/torch/include/ATen/ops/softplus_backward_ops.h 2024-08-08T20:48:49.9161398Z inflating: build/lib/torch/include/ATen/ops/softplus_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9162069Z inflating: build/lib/torch/include/ATen/ops/softplus_cpu_dispatch.h 2024-08-08T20:48:49.9162916Z inflating: build/lib/torch/include/ATen/ops/softplus_cuda_dispatch.h 2024-08-08T20:48:49.9163781Z inflating: build/lib/torch/include/ATen/ops/softplus_meta.h 2024-08-08T20:48:49.9164938Z inflating: build/lib/torch/include/ATen/ops/softplus_meta_dispatch.h 2024-08-08T20:48:49.9165761Z inflating: build/lib/torch/include/ATen/ops/softplus_native.h 2024-08-08T20:48:49.9166823Z inflating: build/lib/torch/include/ATen/ops/softplus_ops.h 2024-08-08T20:48:49.9167824Z inflating: build/lib/torch/include/ATen/ops/softshrink.h 2024-08-08T20:48:49.9168867Z inflating: build/lib/torch/include/ATen/ops/softshrink_backward.h 2024-08-08T20:48:49.9170604Z inflating: build/lib/torch/include/ATen/ops/softshrink_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9171515Z inflating: build/lib/torch/include/ATen/ops/softshrink_backward_cpu_dispatch.h 2024-08-08T20:48:49.9172230Z inflating: build/lib/torch/include/ATen/ops/softshrink_backward_cuda_dispatch.h 2024-08-08T20:48:49.9173032Z inflating: build/lib/torch/include/ATen/ops/softshrink_backward_meta.h 2024-08-08T20:48:49.9175260Z inflating: build/lib/torch/include/ATen/ops/softshrink_backward_meta_dispatch.h 2024-08-08T20:48:49.9175977Z inflating: build/lib/torch/include/ATen/ops/softshrink_backward_native.h 2024-08-08T20:48:49.9176866Z inflating: build/lib/torch/include/ATen/ops/softshrink_backward_ops.h 2024-08-08T20:48:49.9178459Z inflating: build/lib/torch/include/ATen/ops/softshrink_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9179243Z inflating: build/lib/torch/include/ATen/ops/softshrink_cpu_dispatch.h 2024-08-08T20:48:49.9180004Z inflating: build/lib/torch/include/ATen/ops/softshrink_cuda_dispatch.h 2024-08-08T20:48:49.9180792Z inflating: build/lib/torch/include/ATen/ops/softshrink_meta.h 2024-08-08T20:48:49.9182095Z inflating: build/lib/torch/include/ATen/ops/softshrink_meta_dispatch.h 2024-08-08T20:48:49.9182928Z inflating: build/lib/torch/include/ATen/ops/softshrink_native.h 2024-08-08T20:48:49.9183944Z inflating: build/lib/torch/include/ATen/ops/softshrink_ops.h 2024-08-08T20:48:49.9185094Z inflating: build/lib/torch/include/ATen/ops/sort.h 2024-08-08T20:48:49.9186537Z inflating: build/lib/torch/include/ATen/ops/sort_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9187853Z inflating: build/lib/torch/include/ATen/ops/sort_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9188829Z inflating: build/lib/torch/include/ATen/ops/sort_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9189551Z inflating: build/lib/torch/include/ATen/ops/sort_cpu_dispatch.h 2024-08-08T20:48:49.9190299Z inflating: build/lib/torch/include/ATen/ops/sort_cuda_dispatch.h 2024-08-08T20:48:49.9191326Z inflating: build/lib/torch/include/ATen/ops/sort_meta.h 2024-08-08T20:48:49.9192407Z inflating: build/lib/torch/include/ATen/ops/sort_meta_dispatch.h 2024-08-08T20:48:49.9193374Z inflating: build/lib/torch/include/ATen/ops/sort_native.h 2024-08-08T20:48:49.9194728Z inflating: build/lib/torch/include/ATen/ops/sort_ops.h 2024-08-08T20:48:49.9195922Z inflating: build/lib/torch/include/ATen/ops/sparse_bsc_tensor.h 2024-08-08T20:48:49.9197646Z inflating: build/lib/torch/include/ATen/ops/sparse_bsc_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9198308Z inflating: build/lib/torch/include/ATen/ops/sparse_bsc_tensor_native.h 2024-08-08T20:48:49.9199112Z inflating: build/lib/torch/include/ATen/ops/sparse_bsc_tensor_ops.h 2024-08-08T20:48:49.9200190Z inflating: build/lib/torch/include/ATen/ops/sparse_bsr_tensor.h 2024-08-08T20:48:49.9201676Z inflating: build/lib/torch/include/ATen/ops/sparse_bsr_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9202281Z inflating: build/lib/torch/include/ATen/ops/sparse_bsr_tensor_native.h 2024-08-08T20:48:49.9203236Z inflating: build/lib/torch/include/ATen/ops/sparse_bsr_tensor_ops.h 2024-08-08T20:48:49.9204767Z inflating: build/lib/torch/include/ATen/ops/sparse_compressed_tensor.h 2024-08-08T20:48:49.9206432Z inflating: build/lib/torch/include/ATen/ops/sparse_compressed_tensor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9207120Z inflating: build/lib/torch/include/ATen/ops/sparse_compressed_tensor_native.h 2024-08-08T20:48:49.9208255Z inflating: build/lib/torch/include/ATen/ops/sparse_compressed_tensor_ops.h 2024-08-08T20:48:49.9209348Z inflating: build/lib/torch/include/ATen/ops/sparse_coo_tensor.h 2024-08-08T20:48:49.9210686Z inflating: build/lib/torch/include/ATen/ops/sparse_coo_tensor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9211997Z inflating: build/lib/torch/include/ATen/ops/sparse_coo_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9212593Z inflating: build/lib/torch/include/ATen/ops/sparse_coo_tensor_native.h 2024-08-08T20:48:49.9213701Z inflating: build/lib/torch/include/ATen/ops/sparse_coo_tensor_ops.h 2024-08-08T20:48:49.9214785Z inflating: build/lib/torch/include/ATen/ops/sparse_csc_tensor.h 2024-08-08T20:48:49.9216295Z inflating: build/lib/torch/include/ATen/ops/sparse_csc_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9216942Z inflating: build/lib/torch/include/ATen/ops/sparse_csc_tensor_native.h 2024-08-08T20:48:49.9217921Z inflating: build/lib/torch/include/ATen/ops/sparse_csc_tensor_ops.h 2024-08-08T20:48:49.9219017Z inflating: build/lib/torch/include/ATen/ops/sparse_csr_tensor.h 2024-08-08T20:48:49.9220605Z inflating: build/lib/torch/include/ATen/ops/sparse_csr_tensor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9221221Z inflating: build/lib/torch/include/ATen/ops/sparse_csr_tensor_native.h 2024-08-08T20:48:49.9222210Z inflating: build/lib/torch/include/ATen/ops/sparse_csr_tensor_ops.h 2024-08-08T20:48:49.9222968Z inflating: build/lib/torch/include/ATen/ops/sparse_dim.h 2024-08-08T20:48:49.9224470Z inflating: build/lib/torch/include/ATen/ops/sparse_dim_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9224979Z inflating: build/lib/torch/include/ATen/ops/sparse_dim_native.h 2024-08-08T20:48:49.9225685Z inflating: build/lib/torch/include/ATen/ops/sparse_dim_ops.h 2024-08-08T20:48:49.9226705Z inflating: build/lib/torch/include/ATen/ops/sparse_mask.h 2024-08-08T20:48:49.9227920Z inflating: build/lib/torch/include/ATen/ops/sparse_mask_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9228704Z inflating: build/lib/torch/include/ATen/ops/sparse_mask_native.h 2024-08-08T20:48:49.9229614Z inflating: build/lib/torch/include/ATen/ops/sparse_mask_ops.h 2024-08-08T20:48:49.9230722Z inflating: build/lib/torch/include/ATen/ops/sparse_resize.h 2024-08-08T20:48:49.9231893Z inflating: build/lib/torch/include/ATen/ops/sparse_resize_and_clear.h 2024-08-08T20:48:49.9233459Z inflating: build/lib/torch/include/ATen/ops/sparse_resize_and_clear_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9234501Z inflating: build/lib/torch/include/ATen/ops/sparse_resize_and_clear_meta_dispatch.h 2024-08-08T20:48:49.9235482Z inflating: build/lib/torch/include/ATen/ops/sparse_resize_and_clear_native.h 2024-08-08T20:48:49.9236407Z inflating: build/lib/torch/include/ATen/ops/sparse_resize_and_clear_ops.h 2024-08-08T20:48:49.9237946Z inflating: build/lib/torch/include/ATen/ops/sparse_resize_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9238790Z inflating: build/lib/torch/include/ATen/ops/sparse_resize_meta_dispatch.h 2024-08-08T20:48:49.9239353Z inflating: build/lib/torch/include/ATen/ops/sparse_resize_native.h 2024-08-08T20:48:49.9240081Z inflating: build/lib/torch/include/ATen/ops/sparse_resize_ops.h 2024-08-08T20:48:49.9241692Z inflating: build/lib/torch/include/ATen/ops/sparse_sampled_addmm.h 2024-08-08T20:48:49.9243174Z inflating: build/lib/torch/include/ATen/ops/sparse_sampled_addmm_native.h 2024-08-08T20:48:49.9244292Z inflating: build/lib/torch/include/ATen/ops/sparse_sampled_addmm_ops.h 2024-08-08T20:48:49.9245333Z inflating: build/lib/torch/include/ATen/ops/special_airy_ai.h 2024-08-08T20:48:49.9246991Z inflating: build/lib/torch/include/ATen/ops/special_airy_ai_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9247813Z inflating: build/lib/torch/include/ATen/ops/special_airy_ai_cpu_dispatch.h 2024-08-08T20:48:49.9248806Z inflating: build/lib/torch/include/ATen/ops/special_airy_ai_cuda_dispatch.h 2024-08-08T20:48:49.9249644Z inflating: build/lib/torch/include/ATen/ops/special_airy_ai_meta.h 2024-08-08T20:48:49.9250538Z inflating: build/lib/torch/include/ATen/ops/special_airy_ai_meta_dispatch.h 2024-08-08T20:48:49.9251470Z inflating: build/lib/torch/include/ATen/ops/special_airy_ai_native.h 2024-08-08T20:48:49.9252608Z inflating: build/lib/torch/include/ATen/ops/special_airy_ai_ops.h 2024-08-08T20:48:49.9253555Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j0.h 2024-08-08T20:48:49.9255233Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j0_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9256141Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j0_cpu_dispatch.h 2024-08-08T20:48:49.9257001Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j0_cuda_dispatch.h 2024-08-08T20:48:49.9257539Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j0_meta.h 2024-08-08T20:48:49.9258757Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j0_meta_dispatch.h 2024-08-08T20:48:49.9259915Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j0_native.h 2024-08-08T20:48:49.9260927Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j0_ops.h 2024-08-08T20:48:49.9261907Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j1.h 2024-08-08T20:48:49.9263429Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j1_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9264239Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j1_cpu_dispatch.h 2024-08-08T20:48:49.9265126Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j1_cuda_dispatch.h 2024-08-08T20:48:49.9266319Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j1_meta.h 2024-08-08T20:48:49.9267386Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j1_meta_dispatch.h 2024-08-08T20:48:49.9268053Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j1_native.h 2024-08-08T20:48:49.9269115Z inflating: build/lib/torch/include/ATen/ops/special_bessel_j1_ops.h 2024-08-08T20:48:49.9270485Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y0.h 2024-08-08T20:48:49.9272246Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y0_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9273107Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y0_cpu_dispatch.h 2024-08-08T20:48:49.9273979Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y0_cuda_dispatch.h 2024-08-08T20:48:49.9274601Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y0_meta.h 2024-08-08T20:48:49.9275552Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y0_meta_dispatch.h 2024-08-08T20:48:49.9276610Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y0_native.h 2024-08-08T20:48:49.9277494Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y0_ops.h 2024-08-08T20:48:49.9278375Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y1.h 2024-08-08T20:48:49.9279847Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y1_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9280530Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y1_cpu_dispatch.h 2024-08-08T20:48:49.9281508Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y1_cuda_dispatch.h 2024-08-08T20:48:49.9282352Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y1_meta.h 2024-08-08T20:48:49.9283387Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y1_meta_dispatch.h 2024-08-08T20:48:49.9284315Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y1_native.h 2024-08-08T20:48:49.9285330Z inflating: build/lib/torch/include/ATen/ops/special_bessel_y1_ops.h 2024-08-08T20:48:49.9286522Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_t.h 2024-08-08T20:48:49.9287793Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_t_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9288905Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_t_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9289688Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_t_cpu_dispatch.h 2024-08-08T20:48:49.9290692Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_t_cuda_dispatch.h 2024-08-08T20:48:49.9291602Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_t_meta.h 2024-08-08T20:48:49.9292640Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_t_meta_dispatch.h 2024-08-08T20:48:49.9293629Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_t_native.h 2024-08-08T20:48:49.9294828Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_t_ops.h 2024-08-08T20:48:49.9295881Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_u.h 2024-08-08T20:48:49.9297201Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_u_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9298344Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_u_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9299059Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_u_cpu_dispatch.h 2024-08-08T20:48:49.9300035Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_u_cuda_dispatch.h 2024-08-08T20:48:49.9300934Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_u_meta.h 2024-08-08T20:48:49.9302078Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_u_meta_dispatch.h 2024-08-08T20:48:49.9302966Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_u_native.h 2024-08-08T20:48:49.9304194Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_u_ops.h 2024-08-08T20:48:49.9305239Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_v.h 2024-08-08T20:48:49.9306580Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_v_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9307708Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_v_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9308621Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_v_cpu_dispatch.h 2024-08-08T20:48:49.9310612Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_v_cuda_dispatch.h 2024-08-08T20:48:49.9311126Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_v_meta.h 2024-08-08T20:48:49.9312281Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_v_meta_dispatch.h 2024-08-08T20:48:49.9313148Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_v_native.h 2024-08-08T20:48:49.9314305Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_v_ops.h 2024-08-08T20:48:49.9315405Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_w.h 2024-08-08T20:48:49.9316686Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_w_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9317882Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_w_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9318635Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_w_cpu_dispatch.h 2024-08-08T20:48:49.9319648Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_w_cuda_dispatch.h 2024-08-08T20:48:49.9320539Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_w_meta.h 2024-08-08T20:48:49.9321652Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_w_meta_dispatch.h 2024-08-08T20:48:49.9322608Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_w_native.h 2024-08-08T20:48:49.9323752Z inflating: build/lib/torch/include/ATen/ops/special_chebyshev_polynomial_w_ops.h 2024-08-08T20:48:49.9324607Z inflating: build/lib/torch/include/ATen/ops/special_digamma.h 2024-08-08T20:48:49.9325888Z inflating: build/lib/torch/include/ATen/ops/special_digamma_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9326605Z inflating: build/lib/torch/include/ATen/ops/special_digamma_native.h 2024-08-08T20:48:49.9327633Z inflating: build/lib/torch/include/ATen/ops/special_digamma_ops.h 2024-08-08T20:48:49.9328609Z inflating: build/lib/torch/include/ATen/ops/special_entr.h 2024-08-08T20:48:49.9330045Z inflating: build/lib/torch/include/ATen/ops/special_entr_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9330697Z inflating: build/lib/torch/include/ATen/ops/special_entr_cpu_dispatch.h 2024-08-08T20:48:49.9331687Z inflating: build/lib/torch/include/ATen/ops/special_entr_cuda_dispatch.h 2024-08-08T20:48:49.9332575Z inflating: build/lib/torch/include/ATen/ops/special_entr_meta.h 2024-08-08T20:48:49.9333613Z inflating: build/lib/torch/include/ATen/ops/special_entr_meta_dispatch.h 2024-08-08T20:48:49.9334582Z inflating: build/lib/torch/include/ATen/ops/special_entr_native.h 2024-08-08T20:48:49.9335608Z inflating: build/lib/torch/include/ATen/ops/special_entr_ops.h 2024-08-08T20:48:49.9336572Z inflating: build/lib/torch/include/ATen/ops/special_erf.h 2024-08-08T20:48:49.9337889Z inflating: build/lib/torch/include/ATen/ops/special_erf_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9338629Z inflating: build/lib/torch/include/ATen/ops/special_erf_native.h 2024-08-08T20:48:49.9339697Z inflating: build/lib/torch/include/ATen/ops/special_erf_ops.h 2024-08-08T20:48:49.9340617Z inflating: build/lib/torch/include/ATen/ops/special_erfc.h 2024-08-08T20:48:49.9342443Z inflating: build/lib/torch/include/ATen/ops/special_erfc_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9342837Z inflating: build/lib/torch/include/ATen/ops/special_erfc_native.h 2024-08-08T20:48:49.9343798Z inflating: build/lib/torch/include/ATen/ops/special_erfc_ops.h 2024-08-08T20:48:49.9344758Z inflating: build/lib/torch/include/ATen/ops/special_erfcx.h 2024-08-08T20:48:49.9346135Z inflating: build/lib/torch/include/ATen/ops/special_erfcx_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9346845Z inflating: build/lib/torch/include/ATen/ops/special_erfcx_cpu_dispatch.h 2024-08-08T20:48:49.9347855Z inflating: build/lib/torch/include/ATen/ops/special_erfcx_cuda_dispatch.h 2024-08-08T20:48:49.9348691Z inflating: build/lib/torch/include/ATen/ops/special_erfcx_meta.h 2024-08-08T20:48:49.9349893Z inflating: build/lib/torch/include/ATen/ops/special_erfcx_meta_dispatch.h 2024-08-08T20:48:49.9350672Z inflating: build/lib/torch/include/ATen/ops/special_erfcx_native.h 2024-08-08T20:48:49.9351674Z inflating: build/lib/torch/include/ATen/ops/special_erfcx_ops.h 2024-08-08T20:48:49.9352630Z inflating: build/lib/torch/include/ATen/ops/special_erfinv.h 2024-08-08T20:48:49.9353914Z inflating: build/lib/torch/include/ATen/ops/special_erfinv_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9354636Z inflating: build/lib/torch/include/ATen/ops/special_erfinv_native.h 2024-08-08T20:48:49.9355619Z inflating: build/lib/torch/include/ATen/ops/special_erfinv_ops.h 2024-08-08T20:48:49.9356563Z inflating: build/lib/torch/include/ATen/ops/special_exp2.h 2024-08-08T20:48:49.9357832Z inflating: build/lib/torch/include/ATen/ops/special_exp2_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9359015Z inflating: build/lib/torch/include/ATen/ops/special_exp2_native.h 2024-08-08T20:48:49.9361283Z inflating: build/lib/torch/include/ATen/ops/special_exp2_ops.h 2024-08-08T20:48:49.9361893Z inflating: build/lib/torch/include/ATen/ops/special_expit.h 2024-08-08T20:48:49.9363199Z inflating: build/lib/torch/include/ATen/ops/special_expit_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9364011Z inflating: build/lib/torch/include/ATen/ops/special_expit_native.h 2024-08-08T20:48:49.9364922Z inflating: build/lib/torch/include/ATen/ops/special_expit_ops.h 2024-08-08T20:48:49.9365843Z inflating: build/lib/torch/include/ATen/ops/special_expm1.h 2024-08-08T20:48:49.9367163Z inflating: build/lib/torch/include/ATen/ops/special_expm1_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9367850Z inflating: build/lib/torch/include/ATen/ops/special_expm1_native.h 2024-08-08T20:48:49.9368907Z inflating: build/lib/torch/include/ATen/ops/special_expm1_ops.h 2024-08-08T20:48:49.9369935Z inflating: build/lib/torch/include/ATen/ops/special_gammainc.h 2024-08-08T20:48:49.9371205Z inflating: build/lib/torch/include/ATen/ops/special_gammainc_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9371954Z inflating: build/lib/torch/include/ATen/ops/special_gammainc_native.h 2024-08-08T20:48:49.9372936Z inflating: build/lib/torch/include/ATen/ops/special_gammainc_ops.h 2024-08-08T20:48:49.9373928Z inflating: build/lib/torch/include/ATen/ops/special_gammaincc.h 2024-08-08T20:48:49.9375432Z inflating: build/lib/torch/include/ATen/ops/special_gammaincc_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9376614Z inflating: build/lib/torch/include/ATen/ops/special_gammaincc_native.h 2024-08-08T20:48:49.9377662Z inflating: build/lib/torch/include/ATen/ops/special_gammaincc_ops.h 2024-08-08T20:48:49.9378621Z inflating: build/lib/torch/include/ATen/ops/special_gammaln.h 2024-08-08T20:48:49.9380032Z inflating: build/lib/torch/include/ATen/ops/special_gammaln_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9380711Z inflating: build/lib/torch/include/ATen/ops/special_gammaln_native.h 2024-08-08T20:48:49.9381766Z inflating: build/lib/torch/include/ATen/ops/special_gammaln_ops.h 2024-08-08T20:48:49.9382939Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_h.h 2024-08-08T20:48:49.9384286Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_h_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9385644Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_h_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9386406Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_h_cpu_dispatch.h 2024-08-08T20:48:49.9387445Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_h_cuda_dispatch.h 2024-08-08T20:48:49.9388325Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_h_meta.h 2024-08-08T20:48:49.9389528Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_h_meta_dispatch.h 2024-08-08T20:48:49.9390348Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_h_native.h 2024-08-08T20:48:49.9391502Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_h_ops.h 2024-08-08T20:48:49.9392548Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_he.h 2024-08-08T20:48:49.9393904Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_he_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9394991Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_he_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9395781Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_he_cpu_dispatch.h 2024-08-08T20:48:49.9396750Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_he_cuda_dispatch.h 2024-08-08T20:48:49.9397654Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_he_meta.h 2024-08-08T20:48:49.9398815Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_he_meta_dispatch.h 2024-08-08T20:48:49.9399730Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_he_native.h 2024-08-08T20:48:49.9400932Z inflating: build/lib/torch/include/ATen/ops/special_hermite_polynomial_he_ops.h 2024-08-08T20:48:49.9401772Z inflating: build/lib/torch/include/ATen/ops/special_i0.h 2024-08-08T20:48:49.9403072Z inflating: build/lib/torch/include/ATen/ops/special_i0_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9403771Z inflating: build/lib/torch/include/ATen/ops/special_i0_native.h 2024-08-08T20:48:49.9404792Z inflating: build/lib/torch/include/ATen/ops/special_i0_ops.h 2024-08-08T20:48:49.9405756Z inflating: build/lib/torch/include/ATen/ops/special_i0e.h 2024-08-08T20:48:49.9407134Z inflating: build/lib/torch/include/ATen/ops/special_i0e_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9407853Z inflating: build/lib/torch/include/ATen/ops/special_i0e_cpu_dispatch.h 2024-08-08T20:48:49.9408997Z inflating: build/lib/torch/include/ATen/ops/special_i0e_cuda_dispatch.h 2024-08-08T20:48:49.9409856Z inflating: build/lib/torch/include/ATen/ops/special_i0e_meta.h 2024-08-08T20:48:49.9410908Z inflating: build/lib/torch/include/ATen/ops/special_i0e_meta_dispatch.h 2024-08-08T20:48:49.9411802Z inflating: build/lib/torch/include/ATen/ops/special_i0e_native.h 2024-08-08T20:48:49.9412812Z inflating: build/lib/torch/include/ATen/ops/special_i0e_ops.h 2024-08-08T20:48:49.9413775Z inflating: build/lib/torch/include/ATen/ops/special_i1.h 2024-08-08T20:48:49.9415173Z inflating: build/lib/torch/include/ATen/ops/special_i1_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9415899Z inflating: build/lib/torch/include/ATen/ops/special_i1_cpu_dispatch.h 2024-08-08T20:48:49.9416832Z inflating: build/lib/torch/include/ATen/ops/special_i1_cuda_dispatch.h 2024-08-08T20:48:49.9417731Z inflating: build/lib/torch/include/ATen/ops/special_i1_meta.h 2024-08-08T20:48:49.9418737Z inflating: build/lib/torch/include/ATen/ops/special_i1_meta_dispatch.h 2024-08-08T20:48:49.9419616Z inflating: build/lib/torch/include/ATen/ops/special_i1_native.h 2024-08-08T20:48:49.9420661Z inflating: build/lib/torch/include/ATen/ops/special_i1_ops.h 2024-08-08T20:48:49.9421678Z inflating: build/lib/torch/include/ATen/ops/special_i1e.h 2024-08-08T20:48:49.9423117Z inflating: build/lib/torch/include/ATen/ops/special_i1e_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9423775Z inflating: build/lib/torch/include/ATen/ops/special_i1e_cpu_dispatch.h 2024-08-08T20:48:49.9424783Z inflating: build/lib/torch/include/ATen/ops/special_i1e_cuda_dispatch.h 2024-08-08T20:48:49.9425662Z inflating: build/lib/torch/include/ATen/ops/special_i1e_meta.h 2024-08-08T20:48:49.9426672Z inflating: build/lib/torch/include/ATen/ops/special_i1e_meta_dispatch.h 2024-08-08T20:48:49.9427696Z inflating: build/lib/torch/include/ATen/ops/special_i1e_native.h 2024-08-08T20:48:49.9428661Z inflating: build/lib/torch/include/ATen/ops/special_i1e_ops.h 2024-08-08T20:48:49.9429883Z inflating: build/lib/torch/include/ATen/ops/special_laguerre_polynomial_l.h 2024-08-08T20:48:49.9431204Z inflating: build/lib/torch/include/ATen/ops/special_laguerre_polynomial_l_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9432264Z inflating: build/lib/torch/include/ATen/ops/special_laguerre_polynomial_l_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9433046Z inflating: build/lib/torch/include/ATen/ops/special_laguerre_polynomial_l_cpu_dispatch.h 2024-08-08T20:48:49.9434031Z inflating: build/lib/torch/include/ATen/ops/special_laguerre_polynomial_l_cuda_dispatch.h 2024-08-08T20:48:49.9434947Z inflating: build/lib/torch/include/ATen/ops/special_laguerre_polynomial_l_meta.h 2024-08-08T20:48:49.9435994Z inflating: build/lib/torch/include/ATen/ops/special_laguerre_polynomial_l_meta_dispatch.h 2024-08-08T20:48:49.9436981Z inflating: build/lib/torch/include/ATen/ops/special_laguerre_polynomial_l_native.h 2024-08-08T20:48:49.9438181Z inflating: build/lib/torch/include/ATen/ops/special_laguerre_polynomial_l_ops.h 2024-08-08T20:48:49.9439316Z inflating: build/lib/torch/include/ATen/ops/special_legendre_polynomial_p.h 2024-08-08T20:48:49.9441269Z inflating: build/lib/torch/include/ATen/ops/special_legendre_polynomial_p_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9442035Z inflating: build/lib/torch/include/ATen/ops/special_legendre_polynomial_p_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9443379Z inflating: build/lib/torch/include/ATen/ops/special_legendre_polynomial_p_cpu_dispatch.h 2024-08-08T20:48:49.9444359Z inflating: build/lib/torch/include/ATen/ops/special_legendre_polynomial_p_cuda_dispatch.h 2024-08-08T20:48:49.9445263Z inflating: build/lib/torch/include/ATen/ops/special_legendre_polynomial_p_meta.h 2024-08-08T20:48:49.9446430Z inflating: build/lib/torch/include/ATen/ops/special_legendre_polynomial_p_meta_dispatch.h 2024-08-08T20:48:49.9447352Z inflating: build/lib/torch/include/ATen/ops/special_legendre_polynomial_p_native.h 2024-08-08T20:48:49.9448569Z inflating: build/lib/torch/include/ATen/ops/special_legendre_polynomial_p_ops.h 2024-08-08T20:48:49.9449434Z inflating: build/lib/torch/include/ATen/ops/special_log1p.h 2024-08-08T20:48:49.9450671Z inflating: build/lib/torch/include/ATen/ops/special_log1p_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9451405Z inflating: build/lib/torch/include/ATen/ops/special_log1p_native.h 2024-08-08T20:48:49.9452396Z inflating: build/lib/torch/include/ATen/ops/special_log1p_ops.h 2024-08-08T20:48:49.9453444Z inflating: build/lib/torch/include/ATen/ops/special_log_ndtr.h 2024-08-08T20:48:49.9454800Z inflating: build/lib/torch/include/ATen/ops/special_log_ndtr_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9455478Z inflating: build/lib/torch/include/ATen/ops/special_log_ndtr_cpu_dispatch.h 2024-08-08T20:48:49.9456456Z inflating: build/lib/torch/include/ATen/ops/special_log_ndtr_cuda_dispatch.h 2024-08-08T20:48:49.9457311Z inflating: build/lib/torch/include/ATen/ops/special_log_ndtr_meta.h 2024-08-08T20:48:49.9458368Z inflating: build/lib/torch/include/ATen/ops/special_log_ndtr_meta_dispatch.h 2024-08-08T20:48:49.9459533Z inflating: build/lib/torch/include/ATen/ops/special_log_ndtr_native.h 2024-08-08T20:48:49.9460565Z inflating: build/lib/torch/include/ATen/ops/special_log_ndtr_ops.h 2024-08-08T20:48:49.9461530Z inflating: build/lib/torch/include/ATen/ops/special_log_softmax.h 2024-08-08T20:48:49.9462826Z inflating: build/lib/torch/include/ATen/ops/special_log_softmax_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9463527Z inflating: build/lib/torch/include/ATen/ops/special_log_softmax_native.h 2024-08-08T20:48:49.9464641Z inflating: build/lib/torch/include/ATen/ops/special_log_softmax_ops.h 2024-08-08T20:48:49.9465522Z inflating: build/lib/torch/include/ATen/ops/special_logit.h 2024-08-08T20:48:49.9466757Z inflating: build/lib/torch/include/ATen/ops/special_logit_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9467471Z inflating: build/lib/torch/include/ATen/ops/special_logit_native.h 2024-08-08T20:48:49.9468521Z inflating: build/lib/torch/include/ATen/ops/special_logit_ops.h 2024-08-08T20:48:49.9469508Z inflating: build/lib/torch/include/ATen/ops/special_logsumexp.h 2024-08-08T20:48:49.9470799Z inflating: build/lib/torch/include/ATen/ops/special_logsumexp_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9471526Z inflating: build/lib/torch/include/ATen/ops/special_logsumexp_native.h 2024-08-08T20:48:49.9472560Z inflating: build/lib/torch/include/ATen/ops/special_logsumexp_ops.h 2024-08-08T20:48:49.9473644Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i0.h 2024-08-08T20:48:49.9475166Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i0_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9475922Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i0_cpu_dispatch.h 2024-08-08T20:48:49.9476887Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i0_cuda_dispatch.h 2024-08-08T20:48:49.9477754Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i0_meta.h 2024-08-08T20:48:49.9478795Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i0_meta_dispatch.h 2024-08-08T20:48:49.9479761Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i0_native.h 2024-08-08T20:48:49.9480805Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i0_ops.h 2024-08-08T20:48:49.9481734Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i1.h 2024-08-08T20:48:49.9483125Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i1_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9483914Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i1_cpu_dispatch.h 2024-08-08T20:48:49.9484867Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i1_cuda_dispatch.h 2024-08-08T20:48:49.9485784Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i1_meta.h 2024-08-08T20:48:49.9486822Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i1_meta_dispatch.h 2024-08-08T20:48:49.9487724Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i1_native.h 2024-08-08T20:48:49.9488839Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_i1_ops.h 2024-08-08T20:48:49.9489813Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k0.h 2024-08-08T20:48:49.9491167Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k0_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9491891Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k0_cpu_dispatch.h 2024-08-08T20:48:49.9492872Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k0_cuda_dispatch.h 2024-08-08T20:48:49.9493737Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k0_meta.h 2024-08-08T20:48:49.9494779Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k0_meta_dispatch.h 2024-08-08T20:48:49.9495695Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k0_native.h 2024-08-08T20:48:49.9496715Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k0_ops.h 2024-08-08T20:48:49.9497692Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k1.h 2024-08-08T20:48:49.9499116Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k1_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9499906Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k1_cpu_dispatch.h 2024-08-08T20:48:49.9500843Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k1_cuda_dispatch.h 2024-08-08T20:48:49.9501712Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k1_meta.h 2024-08-08T20:48:49.9502773Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k1_meta_dispatch.h 2024-08-08T20:48:49.9503677Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k1_native.h 2024-08-08T20:48:49.9504712Z inflating: build/lib/torch/include/ATen/ops/special_modified_bessel_k1_ops.h 2024-08-08T20:48:49.9505612Z inflating: build/lib/torch/include/ATen/ops/special_multigammaln.h 2024-08-08T20:48:49.9507079Z inflating: build/lib/torch/include/ATen/ops/special_multigammaln_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9508821Z inflating: build/lib/torch/include/ATen/ops/special_multigammaln_native.h 2024-08-08T20:48:49.9509399Z inflating: build/lib/torch/include/ATen/ops/special_multigammaln_ops.h 2024-08-08T20:48:49.9510393Z inflating: build/lib/torch/include/ATen/ops/special_ndtr.h 2024-08-08T20:48:49.9511710Z inflating: build/lib/torch/include/ATen/ops/special_ndtr_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9512427Z inflating: build/lib/torch/include/ATen/ops/special_ndtr_native.h 2024-08-08T20:48:49.9513467Z inflating: build/lib/torch/include/ATen/ops/special_ndtr_ops.h 2024-08-08T20:48:49.9514417Z inflating: build/lib/torch/include/ATen/ops/special_ndtri.h 2024-08-08T20:48:49.9515801Z inflating: build/lib/torch/include/ATen/ops/special_ndtri_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9516472Z inflating: build/lib/torch/include/ATen/ops/special_ndtri_cpu_dispatch.h 2024-08-08T20:48:49.9517455Z inflating: build/lib/torch/include/ATen/ops/special_ndtri_cuda_dispatch.h 2024-08-08T20:48:49.9518383Z inflating: build/lib/torch/include/ATen/ops/special_ndtri_meta.h 2024-08-08T20:48:49.9519450Z inflating: build/lib/torch/include/ATen/ops/special_ndtri_meta_dispatch.h 2024-08-08T20:48:49.9520373Z inflating: build/lib/torch/include/ATen/ops/special_ndtri_native.h 2024-08-08T20:48:49.9521383Z inflating: build/lib/torch/include/ATen/ops/special_ndtri_ops.h 2024-08-08T20:48:49.9522367Z inflating: build/lib/torch/include/ATen/ops/special_polygamma.h 2024-08-08T20:48:49.9523715Z inflating: build/lib/torch/include/ATen/ops/special_polygamma_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9524462Z inflating: build/lib/torch/include/ATen/ops/special_polygamma_native.h 2024-08-08T20:48:49.9525486Z inflating: build/lib/torch/include/ATen/ops/special_polygamma_ops.h 2024-08-08T20:48:49.9526361Z inflating: build/lib/torch/include/ATen/ops/special_psi.h 2024-08-08T20:48:49.9527692Z inflating: build/lib/torch/include/ATen/ops/special_psi_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9528412Z inflating: build/lib/torch/include/ATen/ops/special_psi_native.h 2024-08-08T20:48:49.9529416Z inflating: build/lib/torch/include/ATen/ops/special_psi_ops.h 2024-08-08T20:48:49.9530398Z inflating: build/lib/torch/include/ATen/ops/special_round.h 2024-08-08T20:48:49.9531671Z inflating: build/lib/torch/include/ATen/ops/special_round_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9532414Z inflating: build/lib/torch/include/ATen/ops/special_round_native.h 2024-08-08T20:48:49.9533498Z inflating: build/lib/torch/include/ATen/ops/special_round_ops.h 2024-08-08T20:48:49.9534622Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k0.h 2024-08-08T20:48:49.9535962Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k0_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9536750Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k0_cpu_dispatch.h 2024-08-08T20:48:49.9537843Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k0_cuda_dispatch.h 2024-08-08T20:48:49.9538666Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k0_meta.h 2024-08-08T20:48:49.9539935Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k0_meta_dispatch.h 2024-08-08T20:48:49.9540864Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k0_native.h 2024-08-08T20:48:49.9541883Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k0_ops.h 2024-08-08T20:48:49.9542865Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k1.h 2024-08-08T20:48:49.9544209Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k1_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9544999Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k1_cpu_dispatch.h 2024-08-08T20:48:49.9545991Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k1_cuda_dispatch.h 2024-08-08T20:48:49.9546885Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k1_meta.h 2024-08-08T20:48:49.9547931Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k1_meta_dispatch.h 2024-08-08T20:48:49.9548850Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k1_native.h 2024-08-08T20:48:49.9549886Z inflating: build/lib/torch/include/ATen/ops/special_scaled_modified_bessel_k1_ops.h 2024-08-08T20:48:49.9551060Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_t.h 2024-08-08T20:48:49.9552353Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_t_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9553407Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_t_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9554268Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_t_cpu_dispatch.h 2024-08-08T20:48:49.9555278Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_t_cuda_dispatch.h 2024-08-08T20:48:49.9556121Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_t_meta.h 2024-08-08T20:48:49.9557240Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_t_meta_dispatch.h 2024-08-08T20:48:49.9558159Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_t_native.h 2024-08-08T20:48:49.9562676Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_t_ops.h 2024-08-08T20:48:49.9563501Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_u.h 2024-08-08T20:48:49.9564816Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_u_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9565922Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_u_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9566720Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_u_cpu_dispatch.h 2024-08-08T20:48:49.9567689Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_u_cuda_dispatch.h 2024-08-08T20:48:49.9568663Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_u_meta.h 2024-08-08T20:48:49.9569733Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_u_meta_dispatch.h 2024-08-08T20:48:49.9570677Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_u_native.h 2024-08-08T20:48:49.9571876Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_u_ops.h 2024-08-08T20:48:49.9573021Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_v.h 2024-08-08T20:48:49.9574313Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_v_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9575333Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_v_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9576377Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_v_cpu_dispatch.h 2024-08-08T20:48:49.9578280Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_v_cuda_dispatch.h 2024-08-08T20:48:49.9578851Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_v_meta.h 2024-08-08T20:48:49.9579880Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_v_meta_dispatch.h 2024-08-08T20:48:49.9580906Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_v_native.h 2024-08-08T20:48:49.9582122Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_v_ops.h 2024-08-08T20:48:49.9583232Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_w.h 2024-08-08T20:48:49.9584543Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_w_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9585592Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_w_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9586133Z inflating: build/lib/torch/include/ATen/ops/unbind_ops.h 2024-08-08T20:48:49.9587189Z inflating: build/lib/torch/include/ATen/ops/unflatten.h 2024-08-08T20:48:49.9589081Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_w_cpu_dispatch.h 2024-08-08T20:48:49.9589970Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_w_cuda_dispatch.h 2024-08-08T20:48:49.9590813Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_w_meta.h 2024-08-08T20:48:49.9591916Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_w_meta_dispatch.h 2024-08-08T20:48:49.9592818Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_w_native.h 2024-08-08T20:48:49.9594066Z inflating: build/lib/torch/include/ATen/ops/special_shifted_chebyshev_polynomial_w_ops.h 2024-08-08T20:48:49.9594803Z inflating: build/lib/torch/include/ATen/ops/special_sinc.h 2024-08-08T20:48:49.9596087Z inflating: build/lib/torch/include/ATen/ops/special_sinc_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9596902Z inflating: build/lib/torch/include/ATen/ops/special_sinc_native.h 2024-08-08T20:48:49.9597905Z inflating: build/lib/torch/include/ATen/ops/special_sinc_ops.h 2024-08-08T20:48:49.9598976Z inflating: build/lib/torch/include/ATen/ops/special_softmax.h 2024-08-08T20:48:49.9600192Z inflating: build/lib/torch/include/ATen/ops/special_softmax_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9600907Z inflating: build/lib/torch/include/ATen/ops/special_softmax_native.h 2024-08-08T20:48:49.9601920Z inflating: build/lib/torch/include/ATen/ops/special_softmax_ops.h 2024-08-08T20:48:49.9602939Z inflating: build/lib/torch/include/ATen/ops/special_spherical_bessel_j0.h 2024-08-08T20:48:49.9604271Z inflating: build/lib/torch/include/ATen/ops/special_spherical_bessel_j0_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9605009Z inflating: build/lib/torch/include/ATen/ops/special_spherical_bessel_j0_cpu_dispatch.h 2024-08-08T20:48:49.9606011Z inflating: build/lib/torch/include/ATen/ops/special_spherical_bessel_j0_cuda_dispatch.h 2024-08-08T20:48:49.9606905Z inflating: build/lib/torch/include/ATen/ops/special_spherical_bessel_j0_meta.h 2024-08-08T20:48:49.9608000Z inflating: build/lib/torch/include/ATen/ops/special_spherical_bessel_j0_meta_dispatch.h 2024-08-08T20:48:49.9609005Z inflating: build/lib/torch/include/ATen/ops/special_spherical_bessel_j0_native.h 2024-08-08T20:48:49.9610116Z inflating: build/lib/torch/include/ATen/ops/special_spherical_bessel_j0_ops.h 2024-08-08T20:48:49.9611096Z inflating: build/lib/torch/include/ATen/ops/special_xlog1py.h 2024-08-08T20:48:49.9612416Z inflating: build/lib/torch/include/ATen/ops/special_xlog1py_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9613497Z inflating: build/lib/torch/include/ATen/ops/special_xlog1py_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9614199Z inflating: build/lib/torch/include/ATen/ops/special_xlog1py_cpu_dispatch.h 2024-08-08T20:48:49.9615156Z inflating: build/lib/torch/include/ATen/ops/special_xlog1py_cuda_dispatch.h 2024-08-08T20:48:49.9616044Z inflating: build/lib/torch/include/ATen/ops/special_xlog1py_meta.h 2024-08-08T20:48:49.9617040Z inflating: build/lib/torch/include/ATen/ops/special_xlog1py_meta_dispatch.h 2024-08-08T20:48:49.9617972Z inflating: build/lib/torch/include/ATen/ops/special_xlog1py_native.h 2024-08-08T20:48:49.9619492Z inflating: build/lib/torch/include/ATen/ops/special_xlog1py_ops.h 2024-08-08T20:48:49.9620195Z inflating: build/lib/torch/include/ATen/ops/special_xlogy.h 2024-08-08T20:48:49.9621543Z inflating: build/lib/torch/include/ATen/ops/special_xlogy_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9622256Z inflating: build/lib/torch/include/ATen/ops/special_xlogy_native.h 2024-08-08T20:48:49.9623774Z inflating: build/lib/torch/include/ATen/ops/special_xlogy_ops.h 2024-08-08T20:48:49.9624332Z inflating: build/lib/torch/include/ATen/ops/special_zeta.h 2024-08-08T20:48:49.9625868Z inflating: build/lib/torch/include/ATen/ops/special_zeta_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9626846Z inflating: build/lib/torch/include/ATen/ops/special_zeta_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9627536Z inflating: build/lib/torch/include/ATen/ops/special_zeta_cpu_dispatch.h 2024-08-08T20:48:49.9628505Z inflating: build/lib/torch/include/ATen/ops/special_zeta_cuda_dispatch.h 2024-08-08T20:48:49.9629383Z inflating: build/lib/torch/include/ATen/ops/special_zeta_meta.h 2024-08-08T20:48:49.9630395Z inflating: build/lib/torch/include/ATen/ops/special_zeta_meta_dispatch.h 2024-08-08T20:48:49.9631323Z inflating: build/lib/torch/include/ATen/ops/special_zeta_native.h 2024-08-08T20:48:49.9632659Z inflating: build/lib/torch/include/ATen/ops/special_zeta_ops.h 2024-08-08T20:48:49.9633499Z inflating: build/lib/torch/include/ATen/ops/split.h 2024-08-08T20:48:49.9634817Z inflating: build/lib/torch/include/ATen/ops/split_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9635835Z inflating: build/lib/torch/include/ATen/ops/split_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9636766Z inflating: build/lib/torch/include/ATen/ops/split_copy.h 2024-08-08T20:48:49.9638001Z inflating: build/lib/torch/include/ATen/ops/split_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9639135Z inflating: build/lib/torch/include/ATen/ops/split_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9639748Z inflating: build/lib/torch/include/ATen/ops/split_copy_native.h 2024-08-08T20:48:49.9640759Z inflating: build/lib/torch/include/ATen/ops/split_copy_ops.h 2024-08-08T20:48:49.9641771Z inflating: build/lib/torch/include/ATen/ops/split_native.h 2024-08-08T20:48:49.9643304Z inflating: build/lib/torch/include/ATen/ops/split_ops.h 2024-08-08T20:48:49.9644887Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes.h 2024-08-08T20:48:49.9645729Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9646759Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes_copy.h 2024-08-08T20:48:49.9648125Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9649319Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9650070Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes_copy_cuda_dispatch.h 2024-08-08T20:48:49.9650922Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes_copy_native.h 2024-08-08T20:48:49.9651949Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes_copy_ops.h 2024-08-08T20:48:49.9652945Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes_native.h 2024-08-08T20:48:49.9653921Z inflating: build/lib/torch/include/ATen/ops/split_with_sizes_ops.h 2024-08-08T20:48:49.9654794Z inflating: build/lib/torch/include/ATen/ops/sqrt.h 2024-08-08T20:48:49.9656153Z inflating: build/lib/torch/include/ATen/ops/sqrt_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9656859Z inflating: build/lib/torch/include/ATen/ops/sqrt_cpu_dispatch.h 2024-08-08T20:48:49.9657860Z inflating: build/lib/torch/include/ATen/ops/sqrt_cuda_dispatch.h 2024-08-08T20:48:49.9658894Z inflating: build/lib/torch/include/ATen/ops/sqrt_meta.h 2024-08-08T20:48:49.9660131Z inflating: build/lib/torch/include/ATen/ops/sqrt_meta_dispatch.h 2024-08-08T20:48:49.9660953Z inflating: build/lib/torch/include/ATen/ops/sqrt_native.h 2024-08-08T20:48:49.9662366Z inflating: build/lib/torch/include/ATen/ops/sqrt_ops.h 2024-08-08T20:48:49.9663045Z inflating: build/lib/torch/include/ATen/ops/square.h 2024-08-08T20:48:49.9664363Z inflating: build/lib/torch/include/ATen/ops/square_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9665050Z inflating: build/lib/torch/include/ATen/ops/square_native.h 2024-08-08T20:48:49.9666079Z inflating: build/lib/torch/include/ATen/ops/square_ops.h 2024-08-08T20:48:49.9667055Z inflating: build/lib/torch/include/ATen/ops/squeeze.h 2024-08-08T20:48:49.9668327Z inflating: build/lib/torch/include/ATen/ops/squeeze_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9669308Z inflating: build/lib/torch/include/ATen/ops/squeeze_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9670118Z inflating: build/lib/torch/include/ATen/ops/squeeze_copy.h 2024-08-08T20:48:49.9671407Z inflating: build/lib/torch/include/ATen/ops/squeeze_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9672539Z inflating: build/lib/torch/include/ATen/ops/squeeze_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9673129Z inflating: build/lib/torch/include/ATen/ops/squeeze_copy_native.h 2024-08-08T20:48:49.9674670Z inflating: build/lib/torch/include/ATen/ops/squeeze_copy_ops.h 2024-08-08T20:48:49.9675109Z inflating: build/lib/torch/include/ATen/ops/squeeze_native.h 2024-08-08T20:48:49.9676915Z inflating: build/lib/torch/include/ATen/ops/squeeze_ops.h 2024-08-08T20:48:49.9677596Z inflating: build/lib/torch/include/ATen/ops/sspaddmm.h 2024-08-08T20:48:49.9678878Z inflating: build/lib/torch/include/ATen/ops/sspaddmm_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9679735Z inflating: build/lib/torch/include/ATen/ops/sspaddmm_cpu_dispatch.h 2024-08-08T20:48:49.9680704Z inflating: build/lib/torch/include/ATen/ops/sspaddmm_cuda_dispatch.h 2024-08-08T20:48:49.9681611Z inflating: build/lib/torch/include/ATen/ops/sspaddmm_native.h 2024-08-08T20:48:49.9682676Z inflating: build/lib/torch/include/ATen/ops/sspaddmm_ops.h 2024-08-08T20:48:49.9683587Z inflating: build/lib/torch/include/ATen/ops/stack.h 2024-08-08T20:48:49.9684892Z inflating: build/lib/torch/include/ATen/ops/stack_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9685612Z inflating: build/lib/torch/include/ATen/ops/stack_native.h 2024-08-08T20:48:49.9686637Z inflating: build/lib/torch/include/ATen/ops/stack_ops.h 2024-08-08T20:48:49.9688146Z inflating: build/lib/torch/include/ATen/ops/std.h 2024-08-08T20:48:49.9688973Z inflating: build/lib/torch/include/ATen/ops/std_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9690073Z inflating: build/lib/torch/include/ATen/ops/std_cpu_dispatch.h 2024-08-08T20:48:49.9690958Z inflating: build/lib/torch/include/ATen/ops/std_cuda_dispatch.h 2024-08-08T20:48:49.9692008Z inflating: build/lib/torch/include/ATen/ops/std_mean.h 2024-08-08T20:48:49.9693303Z inflating: build/lib/torch/include/ATen/ops/std_mean_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9694374Z inflating: build/lib/torch/include/ATen/ops/std_mean_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9695223Z inflating: build/lib/torch/include/ATen/ops/std_mean_cpu_dispatch.h 2024-08-08T20:48:49.9696190Z inflating: build/lib/torch/include/ATen/ops/std_mean_cuda_dispatch.h 2024-08-08T20:48:49.9697139Z inflating: build/lib/torch/include/ATen/ops/std_mean_native.h 2024-08-08T20:48:49.9698765Z inflating: build/lib/torch/include/ATen/ops/std_mean_ops.h 2024-08-08T20:48:49.9699340Z inflating: build/lib/torch/include/ATen/ops/std_native.h 2024-08-08T20:48:49.9701074Z inflating: build/lib/torch/include/ATen/ops/std_ops.h 2024-08-08T20:48:49.9701730Z inflating: build/lib/torch/include/ATen/ops/stft.h 2024-08-08T20:48:49.9703141Z inflating: build/lib/torch/include/ATen/ops/stft_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9703868Z inflating: build/lib/torch/include/ATen/ops/stft_native.h 2024-08-08T20:48:49.9705119Z inflating: build/lib/torch/include/ATen/ops/stft_ops.h 2024-08-08T20:48:49.9705940Z inflating: build/lib/torch/include/ATen/ops/stride.h 2024-08-08T20:48:49.9707298Z inflating: build/lib/torch/include/ATen/ops/stride_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9708036Z inflating: build/lib/torch/include/ATen/ops/stride_native.h 2024-08-08T20:48:49.9709031Z inflating: build/lib/torch/include/ATen/ops/stride_ops.h 2024-08-08T20:48:49.9710523Z inflating: build/lib/torch/include/ATen/ops/sub.h 2024-08-08T20:48:49.9712369Z inflating: build/lib/torch/include/ATen/ops/sub_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9713132Z inflating: build/lib/torch/include/ATen/ops/sub_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9713876Z inflating: build/lib/torch/include/ATen/ops/sub_cpu_dispatch.h 2024-08-08T20:48:49.9714873Z inflating: build/lib/torch/include/ATen/ops/sub_cuda_dispatch.h 2024-08-08T20:48:49.9715776Z inflating: build/lib/torch/include/ATen/ops/sub_meta.h 2024-08-08T20:48:49.9716817Z inflating: build/lib/torch/include/ATen/ops/sub_meta_dispatch.h 2024-08-08T20:48:49.9717754Z inflating: build/lib/torch/include/ATen/ops/sub_native.h 2024-08-08T20:48:49.9719301Z inflating: build/lib/torch/include/ATen/ops/sub_ops.h 2024-08-08T20:48:49.9719962Z inflating: build/lib/torch/include/ATen/ops/subtract.h 2024-08-08T20:48:49.9721336Z inflating: build/lib/torch/include/ATen/ops/subtract_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9722064Z inflating: build/lib/torch/include/ATen/ops/subtract_native.h 2024-08-08T20:48:49.9723555Z inflating: build/lib/torch/include/ATen/ops/subtract_ops.h 2024-08-08T20:48:49.9724182Z inflating: build/lib/torch/include/ATen/ops/sum.h 2024-08-08T20:48:49.9725511Z inflating: build/lib/torch/include/ATen/ops/sum_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9726634Z inflating: build/lib/torch/include/ATen/ops/sum_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9727506Z inflating: build/lib/torch/include/ATen/ops/sum_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9728357Z inflating: build/lib/torch/include/ATen/ops/sum_cpu_dispatch.h 2024-08-08T20:48:49.9729353Z inflating: build/lib/torch/include/ATen/ops/sum_cuda_dispatch.h 2024-08-08T20:48:49.9730282Z inflating: build/lib/torch/include/ATen/ops/sum_meta.h 2024-08-08T20:48:49.9731375Z inflating: build/lib/torch/include/ATen/ops/sum_meta_dispatch.h 2024-08-08T20:48:49.9732344Z inflating: build/lib/torch/include/ATen/ops/sum_native.h 2024-08-08T20:48:49.9733900Z inflating: build/lib/torch/include/ATen/ops/sum_ops.h 2024-08-08T20:48:49.9734564Z inflating: build/lib/torch/include/ATen/ops/sum_to_size.h 2024-08-08T20:48:49.9735884Z inflating: build/lib/torch/include/ATen/ops/sum_to_size_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9736641Z inflating: build/lib/torch/include/ATen/ops/sum_to_size_native.h 2024-08-08T20:48:49.9737588Z inflating: build/lib/torch/include/ATen/ops/sum_to_size_ops.h 2024-08-08T20:48:49.9738508Z inflating: build/lib/torch/include/ATen/ops/svd.h 2024-08-08T20:48:49.9739852Z inflating: build/lib/torch/include/ATen/ops/svd_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9740553Z inflating: build/lib/torch/include/ATen/ops/svd_native.h 2024-08-08T20:48:49.9741623Z inflating: build/lib/torch/include/ATen/ops/svd_ops.h 2024-08-08T20:48:49.9742563Z inflating: build/lib/torch/include/ATen/ops/swapaxes.h 2024-08-08T20:48:49.9744377Z inflating: build/lib/torch/include/ATen/ops/swapaxes_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9744749Z inflating: build/lib/torch/include/ATen/ops/swapaxes_native.h 2024-08-08T20:48:49.9745730Z inflating: build/lib/torch/include/ATen/ops/swapaxes_ops.h 2024-08-08T20:48:49.9746652Z inflating: build/lib/torch/include/ATen/ops/swapdims.h 2024-08-08T20:48:49.9747921Z inflating: build/lib/torch/include/ATen/ops/swapdims_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9748669Z inflating: build/lib/torch/include/ATen/ops/swapdims_native.h 2024-08-08T20:48:49.9749714Z inflating: build/lib/torch/include/ATen/ops/swapdims_ops.h 2024-08-08T20:48:49.9750704Z inflating: build/lib/torch/include/ATen/ops/sym_constrain_range.h 2024-08-08T20:48:49.9751995Z inflating: build/lib/torch/include/ATen/ops/sym_constrain_range_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9752742Z inflating: build/lib/torch/include/ATen/ops/sym_constrain_range_for_size.h 2024-08-08T20:48:49.9754033Z inflating: build/lib/torch/include/ATen/ops/sym_constrain_range_for_size_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9754759Z inflating: build/lib/torch/include/ATen/ops/sym_constrain_range_for_size_native.h 2024-08-08T20:48:49.9755780Z inflating: build/lib/torch/include/ATen/ops/sym_constrain_range_for_size_ops.h 2024-08-08T20:48:49.9756698Z inflating: build/lib/torch/include/ATen/ops/sym_constrain_range_native.h 2024-08-08T20:48:49.9757667Z inflating: build/lib/torch/include/ATen/ops/sym_constrain_range_ops.h 2024-08-08T20:48:49.9758862Z inflating: build/lib/torch/include/ATen/ops/sym_numel.h 2024-08-08T20:48:49.9760052Z inflating: build/lib/torch/include/ATen/ops/sym_numel_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9760736Z inflating: build/lib/torch/include/ATen/ops/sym_numel_native.h 2024-08-08T20:48:49.9761728Z inflating: build/lib/torch/include/ATen/ops/sym_numel_ops.h 2024-08-08T20:48:49.9762673Z inflating: build/lib/torch/include/ATen/ops/sym_size.h 2024-08-08T20:48:49.9763967Z inflating: build/lib/torch/include/ATen/ops/sym_size_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9764672Z inflating: build/lib/torch/include/ATen/ops/sym_size_native.h 2024-08-08T20:48:49.9765662Z inflating: build/lib/torch/include/ATen/ops/sym_size_ops.h 2024-08-08T20:48:49.9766638Z inflating: build/lib/torch/include/ATen/ops/sym_storage_offset.h 2024-08-08T20:48:49.9767892Z inflating: build/lib/torch/include/ATen/ops/sym_storage_offset_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9768634Z inflating: build/lib/torch/include/ATen/ops/sym_storage_offset_native.h 2024-08-08T20:48:49.9769571Z inflating: build/lib/torch/include/ATen/ops/sym_storage_offset_ops.h 2024-08-08T20:48:49.9770440Z inflating: build/lib/torch/include/ATen/ops/sym_stride.h 2024-08-08T20:48:49.9771782Z inflating: build/lib/torch/include/ATen/ops/sym_stride_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9772551Z inflating: build/lib/torch/include/ATen/ops/sym_stride_native.h 2024-08-08T20:48:49.9773649Z inflating: build/lib/torch/include/ATen/ops/sym_stride_ops.h 2024-08-08T20:48:49.9774406Z inflating: build/lib/torch/include/ATen/ops/t.h 2024-08-08T20:48:49.9776168Z inflating: build/lib/torch/include/ATen/ops/t_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9777142Z inflating: build/lib/torch/include/ATen/ops/t_copy.h 2024-08-08T20:48:49.9778384Z inflating: build/lib/torch/include/ATen/ops/t_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9779510Z inflating: build/lib/torch/include/ATen/ops/t_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9780161Z inflating: build/lib/torch/include/ATen/ops/t_copy_native.h 2024-08-08T20:48:49.9781226Z inflating: build/lib/torch/include/ATen/ops/t_copy_ops.h 2024-08-08T20:48:49.9782158Z inflating: build/lib/torch/include/ATen/ops/t_native.h 2024-08-08T20:48:49.9783159Z inflating: build/lib/torch/include/ATen/ops/t_ops.h 2024-08-08T20:48:49.9784116Z inflating: build/lib/torch/include/ATen/ops/take.h 2024-08-08T20:48:49.9785684Z inflating: build/lib/torch/include/ATen/ops/take_along_dim.h 2024-08-08T20:48:49.9786517Z inflating: build/lib/torch/include/ATen/ops/take_along_dim_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9787369Z inflating: build/lib/torch/include/ATen/ops/take_along_dim_native.h 2024-08-08T20:48:49.9788371Z inflating: build/lib/torch/include/ATen/ops/take_along_dim_ops.h 2024-08-08T20:48:49.9789424Z inflating: build/lib/torch/include/ATen/ops/take_cpu_dispatch.h 2024-08-08T20:48:49.9790342Z inflating: build/lib/torch/include/ATen/ops/take_cuda_dispatch.h 2024-08-08T20:48:49.9791225Z inflating: build/lib/torch/include/ATen/ops/take_native.h 2024-08-08T20:48:49.9792273Z inflating: build/lib/torch/include/ATen/ops/take_ops.h 2024-08-08T20:48:49.9793196Z inflating: build/lib/torch/include/ATen/ops/tan.h 2024-08-08T20:48:49.9794593Z inflating: build/lib/torch/include/ATen/ops/tan_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9795320Z inflating: build/lib/torch/include/ATen/ops/tan_cpu_dispatch.h 2024-08-08T20:48:49.9796338Z inflating: build/lib/torch/include/ATen/ops/tan_cuda_dispatch.h 2024-08-08T20:48:49.9797190Z inflating: build/lib/torch/include/ATen/ops/tan_meta.h 2024-08-08T20:48:49.9798300Z inflating: build/lib/torch/include/ATen/ops/tan_meta_dispatch.h 2024-08-08T20:48:49.9799237Z inflating: build/lib/torch/include/ATen/ops/tan_native.h 2024-08-08T20:48:49.9800261Z inflating: build/lib/torch/include/ATen/ops/tan_ops.h 2024-08-08T20:48:49.9801218Z inflating: build/lib/torch/include/ATen/ops/tanh.h 2024-08-08T20:48:49.9802293Z inflating: build/lib/torch/include/ATen/ops/tanh_backward.h 2024-08-08T20:48:49.9803701Z inflating: build/lib/torch/include/ATen/ops/tanh_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9804453Z inflating: build/lib/torch/include/ATen/ops/tanh_backward_cpu_dispatch.h 2024-08-08T20:48:49.9805382Z inflating: build/lib/torch/include/ATen/ops/tanh_backward_cuda_dispatch.h 2024-08-08T20:48:49.9806258Z inflating: build/lib/torch/include/ATen/ops/tanh_backward_meta.h 2024-08-08T20:48:49.9807317Z inflating: build/lib/torch/include/ATen/ops/tanh_backward_meta_dispatch.h 2024-08-08T20:48:49.9808388Z inflating: build/lib/torch/include/ATen/ops/tanh_backward_native.h 2024-08-08T20:48:49.9809431Z inflating: build/lib/torch/include/ATen/ops/tanh_backward_ops.h 2024-08-08T20:48:49.9810732Z inflating: build/lib/torch/include/ATen/ops/tanh_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9811481Z inflating: build/lib/torch/include/ATen/ops/tanh_cpu_dispatch.h 2024-08-08T20:48:49.9812432Z inflating: build/lib/torch/include/ATen/ops/tanh_cuda_dispatch.h 2024-08-08T20:48:49.9813311Z inflating: build/lib/torch/include/ATen/ops/tanh_meta.h 2024-08-08T20:48:49.9814405Z inflating: build/lib/torch/include/ATen/ops/tanh_meta_dispatch.h 2024-08-08T20:48:49.9815518Z inflating: build/lib/torch/include/ATen/ops/tanh_native.h 2024-08-08T20:48:49.9816467Z inflating: build/lib/torch/include/ATen/ops/tanh_ops.h 2024-08-08T20:48:49.9817589Z inflating: build/lib/torch/include/ATen/ops/tensor_split.h 2024-08-08T20:48:49.9818909Z inflating: build/lib/torch/include/ATen/ops/tensor_split_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9819651Z inflating: build/lib/torch/include/ATen/ops/tensor_split_native.h 2024-08-08T20:48:49.9820704Z inflating: build/lib/torch/include/ATen/ops/tensor_split_ops.h 2024-08-08T20:48:49.9821660Z inflating: build/lib/torch/include/ATen/ops/tensordot.h 2024-08-08T20:48:49.9822967Z inflating: build/lib/torch/include/ATen/ops/tensordot_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9823697Z inflating: build/lib/torch/include/ATen/ops/tensordot_native.h 2024-08-08T20:48:49.9824770Z inflating: build/lib/torch/include/ATen/ops/tensordot_ops.h 2024-08-08T20:48:49.9826381Z inflating: build/lib/torch/include/ATen/ops/thnn_conv2d.h 2024-08-08T20:48:49.9827384Z inflating: build/lib/torch/include/ATen/ops/thnn_conv2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9828144Z inflating: build/lib/torch/include/ATen/ops/thnn_conv2d_native.h 2024-08-08T20:48:49.9829174Z inflating: build/lib/torch/include/ATen/ops/thnn_conv2d_ops.h 2024-08-08T20:48:49.9830155Z inflating: build/lib/torch/include/ATen/ops/threshold.h 2024-08-08T20:48:49.9831225Z inflating: build/lib/torch/include/ATen/ops/threshold_backward.h 2024-08-08T20:48:49.9832659Z inflating: build/lib/torch/include/ATen/ops/threshold_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9833363Z inflating: build/lib/torch/include/ATen/ops/threshold_backward_cpu_dispatch.h 2024-08-08T20:48:49.9834365Z inflating: build/lib/torch/include/ATen/ops/threshold_backward_cuda_dispatch.h 2024-08-08T20:48:49.9835353Z inflating: build/lib/torch/include/ATen/ops/threshold_backward_meta.h 2024-08-08T20:48:49.9836425Z inflating: build/lib/torch/include/ATen/ops/threshold_backward_meta_dispatch.h 2024-08-08T20:48:49.9837398Z inflating: build/lib/torch/include/ATen/ops/threshold_backward_native.h 2024-08-08T20:48:49.9838438Z inflating: build/lib/torch/include/ATen/ops/threshold_backward_ops.h 2024-08-08T20:48:49.9839822Z inflating: build/lib/torch/include/ATen/ops/threshold_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9840508Z inflating: build/lib/torch/include/ATen/ops/threshold_cpu_dispatch.h 2024-08-08T20:48:49.9841708Z inflating: build/lib/torch/include/ATen/ops/threshold_cuda_dispatch.h 2024-08-08T20:48:49.9843436Z inflating: build/lib/torch/include/ATen/ops/threshold_meta.h 2024-08-08T20:48:49.9844024Z inflating: build/lib/torch/include/ATen/ops/threshold_meta_dispatch.h 2024-08-08T20:48:49.9845122Z inflating: build/lib/torch/include/ATen/ops/threshold_native.h 2024-08-08T20:48:49.9846186Z inflating: build/lib/torch/include/ATen/ops/threshold_ops.h 2024-08-08T20:48:49.9847119Z inflating: build/lib/torch/include/ATen/ops/tile.h 2024-08-08T20:48:49.9848438Z inflating: build/lib/torch/include/ATen/ops/tile_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9849121Z inflating: build/lib/torch/include/ATen/ops/tile_native.h 2024-08-08T20:48:49.9850112Z inflating: build/lib/torch/include/ATen/ops/tile_ops.h 2024-08-08T20:48:49.9850998Z inflating: build/lib/torch/include/ATen/ops/to.h 2024-08-08T20:48:49.9852453Z inflating: build/lib/torch/include/ATen/ops/to_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9853062Z inflating: build/lib/torch/include/ATen/ops/to_dense.h 2024-08-08T20:48:49.9854120Z inflating: build/lib/torch/include/ATen/ops/to_dense_backward.h 2024-08-08T20:48:49.9855390Z inflating: build/lib/torch/include/ATen/ops/to_dense_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9856146Z inflating: build/lib/torch/include/ATen/ops/to_dense_backward_native.h 2024-08-08T20:48:49.9857462Z inflating: build/lib/torch/include/ATen/ops/to_dense_backward_ops.h 2024-08-08T20:48:49.9858632Z inflating: build/lib/torch/include/ATen/ops/to_dense_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9859447Z inflating: build/lib/torch/include/ATen/ops/to_dense_native.h 2024-08-08T20:48:49.9860391Z inflating: build/lib/torch/include/ATen/ops/to_dense_ops.h 2024-08-08T20:48:49.9861432Z inflating: build/lib/torch/include/ATen/ops/to_mkldnn.h 2024-08-08T20:48:49.9862419Z inflating: build/lib/torch/include/ATen/ops/to_mkldnn_backward.h 2024-08-08T20:48:49.9863757Z inflating: build/lib/torch/include/ATen/ops/to_mkldnn_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9864791Z inflating: build/lib/torch/include/ATen/ops/to_mkldnn_backward_native.h 2024-08-08T20:48:49.9865963Z inflating: build/lib/torch/include/ATen/ops/to_mkldnn_backward_ops.h 2024-08-08T20:48:49.9867229Z inflating: build/lib/torch/include/ATen/ops/to_mkldnn_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9868214Z inflating: build/lib/torch/include/ATen/ops/to_mkldnn_cpu_dispatch.h 2024-08-08T20:48:49.9869171Z inflating: build/lib/torch/include/ATen/ops/to_mkldnn_native.h 2024-08-08T20:48:49.9870174Z inflating: build/lib/torch/include/ATen/ops/to_mkldnn_ops.h 2024-08-08T20:48:49.9871161Z inflating: build/lib/torch/include/ATen/ops/to_native.h 2024-08-08T20:48:49.9872771Z inflating: build/lib/torch/include/ATen/ops/to_ops.h 2024-08-08T20:48:49.9873610Z inflating: build/lib/torch/include/ATen/ops/to_padded_tensor.h 2024-08-08T20:48:49.9874938Z inflating: build/lib/torch/include/ATen/ops/to_padded_tensor_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9875861Z inflating: build/lib/torch/include/ATen/ops/to_padded_tensor_native.h 2024-08-08T20:48:49.9876916Z inflating: build/lib/torch/include/ATen/ops/to_padded_tensor_ops.h 2024-08-08T20:48:49.9877779Z inflating: build/lib/torch/include/ATen/ops/to_sparse.h 2024-08-08T20:48:49.9878753Z inflating: build/lib/torch/include/ATen/ops/to_sparse_bsc.h 2024-08-08T20:48:49.9880080Z inflating: build/lib/torch/include/ATen/ops/to_sparse_bsc_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9880785Z inflating: build/lib/torch/include/ATen/ops/to_sparse_bsc_native.h 2024-08-08T20:48:49.9881811Z inflating: build/lib/torch/include/ATen/ops/to_sparse_bsc_ops.h 2024-08-08T20:48:49.9882683Z inflating: build/lib/torch/include/ATen/ops/to_sparse_bsr.h 2024-08-08T20:48:49.9883962Z inflating: build/lib/torch/include/ATen/ops/to_sparse_bsr_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9884729Z inflating: build/lib/torch/include/ATen/ops/to_sparse_bsr_native.h 2024-08-08T20:48:49.9885684Z inflating: build/lib/torch/include/ATen/ops/to_sparse_bsr_ops.h 2024-08-08T20:48:49.9887020Z inflating: build/lib/torch/include/ATen/ops/to_sparse_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9887641Z inflating: build/lib/torch/include/ATen/ops/to_sparse_csc.h 2024-08-08T20:48:49.9888938Z inflating: build/lib/torch/include/ATen/ops/to_sparse_csc_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9889711Z inflating: build/lib/torch/include/ATen/ops/to_sparse_csc_native.h 2024-08-08T20:48:49.9890656Z inflating: build/lib/torch/include/ATen/ops/to_sparse_csc_ops.h 2024-08-08T20:48:49.9891567Z inflating: build/lib/torch/include/ATen/ops/to_sparse_csr.h 2024-08-08T20:48:49.9892819Z inflating: build/lib/torch/include/ATen/ops/to_sparse_csr_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9893543Z inflating: build/lib/torch/include/ATen/ops/to_sparse_csr_native.h 2024-08-08T20:48:49.9894531Z inflating: build/lib/torch/include/ATen/ops/to_sparse_csr_ops.h 2024-08-08T20:48:49.9895459Z inflating: build/lib/torch/include/ATen/ops/to_sparse_native.h 2024-08-08T20:48:49.9896519Z inflating: build/lib/torch/include/ATen/ops/to_sparse_ops.h 2024-08-08T20:48:49.9897995Z inflating: build/lib/torch/include/ATen/ops/topk.h 2024-08-08T20:48:49.9899065Z inflating: build/lib/torch/include/ATen/ops/topk_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9899925Z inflating: build/lib/torch/include/ATen/ops/topk_cpu_dispatch.h 2024-08-08T20:48:49.9900999Z inflating: build/lib/torch/include/ATen/ops/topk_cuda_dispatch.h 2024-08-08T20:48:49.9901900Z inflating: build/lib/torch/include/ATen/ops/topk_meta.h 2024-08-08T20:48:49.9903057Z inflating: build/lib/torch/include/ATen/ops/topk_meta_dispatch.h 2024-08-08T20:48:49.9903977Z inflating: build/lib/torch/include/ATen/ops/topk_native.h 2024-08-08T20:48:49.9905072Z inflating: build/lib/torch/include/ATen/ops/topk_ops.h 2024-08-08T20:48:49.9906009Z inflating: build/lib/torch/include/ATen/ops/trace.h 2024-08-08T20:48:49.9907100Z inflating: build/lib/torch/include/ATen/ops/trace_backward.h 2024-08-08T20:48:49.9908670Z inflating: build/lib/torch/include/ATen/ops/trace_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9910255Z inflating: build/lib/torch/include/ATen/ops/trace_backward_native.h 2024-08-08T20:48:49.9910751Z inflating: build/lib/torch/include/ATen/ops/trace_backward_ops.h 2024-08-08T20:48:49.9912055Z inflating: build/lib/torch/include/ATen/ops/trace_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9912899Z inflating: build/lib/torch/include/ATen/ops/trace_cpu_dispatch.h 2024-08-08T20:48:49.9913872Z inflating: build/lib/torch/include/ATen/ops/trace_cuda_dispatch.h 2024-08-08T20:48:49.9914766Z inflating: build/lib/torch/include/ATen/ops/trace_native.h 2024-08-08T20:48:49.9915808Z inflating: build/lib/torch/include/ATen/ops/trace_ops.h 2024-08-08T20:48:49.9916798Z inflating: build/lib/torch/include/ATen/ops/transpose.h 2024-08-08T20:48:49.9918128Z inflating: build/lib/torch/include/ATen/ops/transpose_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9919094Z inflating: build/lib/torch/include/ATen/ops/transpose_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9919863Z inflating: build/lib/torch/include/ATen/ops/transpose_copy.h 2024-08-08T20:48:49.9921196Z inflating: build/lib/torch/include/ATen/ops/transpose_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9922304Z inflating: build/lib/torch/include/ATen/ops/transpose_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9922920Z inflating: build/lib/torch/include/ATen/ops/transpose_copy_native.h 2024-08-08T20:48:49.9923968Z inflating: build/lib/torch/include/ATen/ops/transpose_copy_ops.h 2024-08-08T20:48:49.9924921Z inflating: build/lib/torch/include/ATen/ops/transpose_native.h 2024-08-08T20:48:49.9925962Z inflating: build/lib/torch/include/ATen/ops/transpose_ops.h 2024-08-08T20:48:49.9926898Z inflating: build/lib/torch/include/ATen/ops/trapezoid.h 2024-08-08T20:48:49.9928246Z inflating: build/lib/torch/include/ATen/ops/trapezoid_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9928910Z inflating: build/lib/torch/include/ATen/ops/trapezoid_native.h 2024-08-08T20:48:49.9929993Z inflating: build/lib/torch/include/ATen/ops/trapezoid_ops.h 2024-08-08T20:48:49.9930902Z inflating: build/lib/torch/include/ATen/ops/trapz.h 2024-08-08T20:48:49.9932216Z inflating: build/lib/torch/include/ATen/ops/trapz_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9932946Z inflating: build/lib/torch/include/ATen/ops/trapz_native.h 2024-08-08T20:48:49.9933941Z inflating: build/lib/torch/include/ATen/ops/trapz_ops.h 2024-08-08T20:48:49.9935071Z inflating: build/lib/torch/include/ATen/ops/triangular_solve.h 2024-08-08T20:48:49.9936477Z inflating: build/lib/torch/include/ATen/ops/triangular_solve_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9937224Z inflating: build/lib/torch/include/ATen/ops/triangular_solve_cpu_dispatch.h 2024-08-08T20:48:49.9938250Z inflating: build/lib/torch/include/ATen/ops/triangular_solve_cuda_dispatch.h 2024-08-08T20:48:49.9939188Z inflating: build/lib/torch/include/ATen/ops/triangular_solve_meta.h 2024-08-08T20:48:49.9940241Z inflating: build/lib/torch/include/ATen/ops/triangular_solve_meta_dispatch.h 2024-08-08T20:48:49.9941340Z inflating: build/lib/torch/include/ATen/ops/triangular_solve_native.h 2024-08-08T20:48:49.9942476Z inflating: build/lib/torch/include/ATen/ops/triangular_solve_ops.h 2024-08-08T20:48:49.9943346Z inflating: build/lib/torch/include/ATen/ops/tril.h 2024-08-08T20:48:49.9944737Z inflating: build/lib/torch/include/ATen/ops/tril_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9945464Z inflating: build/lib/torch/include/ATen/ops/tril_cpu_dispatch.h 2024-08-08T20:48:49.9946575Z inflating: build/lib/torch/include/ATen/ops/tril_cuda_dispatch.h 2024-08-08T20:48:49.9947599Z inflating: build/lib/torch/include/ATen/ops/tril_indices.h 2024-08-08T20:48:49.9948900Z inflating: build/lib/torch/include/ATen/ops/tril_indices_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9949710Z inflating: build/lib/torch/include/ATen/ops/tril_indices_cpu_dispatch.h 2024-08-08T20:48:49.9950738Z inflating: build/lib/torch/include/ATen/ops/tril_indices_cuda_dispatch.h 2024-08-08T20:48:49.9951626Z inflating: build/lib/torch/include/ATen/ops/tril_indices_native.h 2024-08-08T20:48:49.9952699Z inflating: build/lib/torch/include/ATen/ops/tril_indices_ops.h 2024-08-08T20:48:49.9953604Z inflating: build/lib/torch/include/ATen/ops/tril_meta.h 2024-08-08T20:48:49.9954674Z inflating: build/lib/torch/include/ATen/ops/tril_meta_dispatch.h 2024-08-08T20:48:49.9955589Z inflating: build/lib/torch/include/ATen/ops/tril_native.h 2024-08-08T20:48:49.9956634Z inflating: build/lib/torch/include/ATen/ops/tril_ops.h 2024-08-08T20:48:49.9957705Z inflating: build/lib/torch/include/ATen/ops/triplet_margin_loss.h 2024-08-08T20:48:49.9960052Z inflating: build/lib/torch/include/ATen/ops/triplet_margin_loss_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9960734Z inflating: build/lib/torch/include/ATen/ops/triplet_margin_loss_native.h 2024-08-08T20:48:49.9961174Z inflating: build/lib/torch/include/ATen/ops/triplet_margin_loss_ops.h 2024-08-08T20:48:49.9962152Z inflating: build/lib/torch/include/ATen/ops/triu.h 2024-08-08T20:48:49.9963490Z inflating: build/lib/torch/include/ATen/ops/triu_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9964281Z inflating: build/lib/torch/include/ATen/ops/triu_cpu_dispatch.h 2024-08-08T20:48:49.9965250Z inflating: build/lib/torch/include/ATen/ops/triu_cuda_dispatch.h 2024-08-08T20:48:49.9966323Z inflating: build/lib/torch/include/ATen/ops/triu_indices.h 2024-08-08T20:48:49.9967537Z inflating: build/lib/torch/include/ATen/ops/triu_indices_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9968455Z inflating: build/lib/torch/include/ATen/ops/triu_indices_cpu_dispatch.h 2024-08-08T20:48:49.9969402Z inflating: build/lib/torch/include/ATen/ops/triu_indices_cuda_dispatch.h 2024-08-08T20:48:49.9970407Z inflating: build/lib/torch/include/ATen/ops/triu_indices_native.h 2024-08-08T20:48:49.9971472Z inflating: build/lib/torch/include/ATen/ops/triu_indices_ops.h 2024-08-08T20:48:49.9972238Z inflating: build/lib/torch/include/ATen/ops/triu_meta.h 2024-08-08T20:48:49.9973418Z inflating: build/lib/torch/include/ATen/ops/triu_meta_dispatch.h 2024-08-08T20:48:49.9974476Z inflating: build/lib/torch/include/ATen/ops/triu_native.h 2024-08-08T20:48:49.9976417Z inflating: build/lib/torch/include/ATen/ops/triu_ops.h 2024-08-08T20:48:49.9976996Z inflating: build/lib/torch/include/ATen/ops/true_divide.h 2024-08-08T20:48:49.9978392Z inflating: build/lib/torch/include/ATen/ops/true_divide_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9979468Z inflating: build/lib/torch/include/ATen/ops/true_divide_native.h 2024-08-08T20:48:49.9980689Z inflating: build/lib/torch/include/ATen/ops/true_divide_ops.h 2024-08-08T20:48:49.9981487Z inflating: build/lib/torch/include/ATen/ops/trunc.h 2024-08-08T20:48:49.9983425Z inflating: build/lib/torch/include/ATen/ops/trunc_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:49.9984137Z inflating: build/lib/torch/include/ATen/ops/trunc_cpu_dispatch.h 2024-08-08T20:48:49.9984698Z inflating: build/lib/torch/include/ATen/ops/trunc_cuda_dispatch.h 2024-08-08T20:48:49.9985234Z inflating: build/lib/torch/include/ATen/ops/trunc_meta.h 2024-08-08T20:48:49.9986386Z inflating: build/lib/torch/include/ATen/ops/trunc_meta_dispatch.h 2024-08-08T20:48:49.9987240Z inflating: build/lib/torch/include/ATen/ops/trunc_native.h 2024-08-08T20:48:49.9988346Z inflating: build/lib/torch/include/ATen/ops/trunc_ops.h 2024-08-08T20:48:49.9989198Z inflating: build/lib/torch/include/ATen/ops/type_as.h 2024-08-08T20:48:49.9991375Z inflating: build/lib/torch/include/ATen/ops/type_as_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9991721Z inflating: build/lib/torch/include/ATen/ops/type_as_native.h 2024-08-08T20:48:49.9992563Z inflating: build/lib/torch/include/ATen/ops/type_as_ops.h 2024-08-08T20:48:49.9993664Z inflating: build/lib/torch/include/ATen/ops/unbind.h 2024-08-08T20:48:49.9995077Z inflating: build/lib/torch/include/ATen/ops/unbind_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9996140Z inflating: build/lib/torch/include/ATen/ops/unbind_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:49.9996849Z inflating: build/lib/torch/include/ATen/ops/unbind_copy.h 2024-08-08T20:48:49.9998192Z inflating: build/lib/torch/include/ATen/ops/unbind_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:49.9999673Z inflating: build/lib/torch/include/ATen/ops/unbind_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0000252Z inflating: build/lib/torch/include/ATen/ops/unbind_copy_native.h 2024-08-08T20:48:50.0001002Z inflating: build/lib/torch/include/ATen/ops/unbind_copy_ops.h 2024-08-08T20:48:50.0001648Z inflating: build/lib/torch/include/ATen/ops/unbind_native.h 2024-08-08T20:48:50.0003512Z inflating: build/lib/torch/include/ATen/ops/unflatten_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0004120Z inflating: build/lib/torch/include/ATen/ops/unflatten_dense_tensors.h 2024-08-08T20:48:50.0005426Z inflating: build/lib/torch/include/ATen/ops/unflatten_dense_tensors_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0006426Z inflating: build/lib/torch/include/ATen/ops/unflatten_dense_tensors_native.h 2024-08-08T20:48:50.0007264Z inflating: build/lib/torch/include/ATen/ops/unflatten_dense_tensors_ops.h 2024-08-08T20:48:50.0008226Z inflating: build/lib/torch/include/ATen/ops/unflatten_native.h 2024-08-08T20:48:50.0009276Z inflating: build/lib/torch/include/ATen/ops/unflatten_ops.h 2024-08-08T20:48:50.0010098Z inflating: build/lib/torch/include/ATen/ops/unfold.h 2024-08-08T20:48:50.0011495Z inflating: build/lib/torch/include/ATen/ops/unfold_backward.h 2024-08-08T20:48:50.0012939Z inflating: build/lib/torch/include/ATen/ops/unfold_backward_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0013727Z inflating: build/lib/torch/include/ATen/ops/unfold_backward_cpu_dispatch.h 2024-08-08T20:48:50.0014735Z inflating: build/lib/torch/include/ATen/ops/unfold_backward_cuda_dispatch.h 2024-08-08T20:48:50.0015543Z inflating: build/lib/torch/include/ATen/ops/unfold_backward_native.h 2024-08-08T20:48:50.0016502Z inflating: build/lib/torch/include/ATen/ops/unfold_backward_ops.h 2024-08-08T20:48:50.0017391Z inflating: build/lib/torch/include/ATen/ops/unfold_copy.h 2024-08-08T20:48:50.0018844Z inflating: build/lib/torch/include/ATen/ops/unfold_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0020364Z inflating: build/lib/torch/include/ATen/ops/unfold_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0020944Z inflating: build/lib/torch/include/ATen/ops/unfold_copy_native.h 2024-08-08T20:48:50.0021471Z inflating: build/lib/torch/include/ATen/ops/unfold_copy_ops.h 2024-08-08T20:48:50.0022305Z inflating: build/lib/torch/include/ATen/ops/unfold_cpu_dispatch.h 2024-08-08T20:48:50.0023287Z inflating: build/lib/torch/include/ATen/ops/unfold_cuda_dispatch.h 2024-08-08T20:48:50.0024114Z inflating: build/lib/torch/include/ATen/ops/unfold_meta_dispatch.h 2024-08-08T20:48:50.0025152Z inflating: build/lib/torch/include/ATen/ops/unfold_native.h 2024-08-08T20:48:50.0026025Z inflating: build/lib/torch/include/ATen/ops/unfold_ops.h 2024-08-08T20:48:50.0027114Z inflating: build/lib/torch/include/ATen/ops/uniform.h 2024-08-08T20:48:50.0028460Z inflating: build/lib/torch/include/ATen/ops/uniform_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0029413Z inflating: build/lib/torch/include/ATen/ops/uniform_cpu_dispatch.h 2024-08-08T20:48:50.0030611Z inflating: build/lib/torch/include/ATen/ops/uniform_cuda_dispatch.h 2024-08-08T20:48:50.0031203Z inflating: build/lib/torch/include/ATen/ops/uniform_meta_dispatch.h 2024-08-08T20:48:50.0032101Z inflating: build/lib/torch/include/ATen/ops/uniform_native.h 2024-08-08T20:48:50.0033211Z inflating: build/lib/torch/include/ATen/ops/uniform_ops.h 2024-08-08T20:48:50.0034259Z inflating: build/lib/torch/include/ATen/ops/unique_consecutive.h 2024-08-08T20:48:50.0035649Z inflating: build/lib/torch/include/ATen/ops/unique_consecutive_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0036376Z inflating: build/lib/torch/include/ATen/ops/unique_consecutive_cpu_dispatch.h 2024-08-08T20:48:50.0037487Z inflating: build/lib/torch/include/ATen/ops/unique_consecutive_cuda_dispatch.h 2024-08-08T20:48:50.0038311Z inflating: build/lib/torch/include/ATen/ops/unique_consecutive_native.h 2024-08-08T20:48:50.0039474Z inflating: build/lib/torch/include/ATen/ops/unique_consecutive_ops.h 2024-08-08T20:48:50.0040593Z inflating: build/lib/torch/include/ATen/ops/unique_dim.h 2024-08-08T20:48:50.0042802Z inflating: build/lib/torch/include/ATen/ops/unique_dim_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0043693Z inflating: build/lib/torch/include/ATen/ops/unique_dim_consecutive.h 2024-08-08T20:48:50.0045074Z inflating: build/lib/torch/include/ATen/ops/unique_dim_consecutive_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0045764Z inflating: build/lib/torch/include/ATen/ops/unique_dim_consecutive_cpu_dispatch.h 2024-08-08T20:48:50.0046419Z inflating: build/lib/torch/include/ATen/ops/unique_dim_consecutive_cuda_dispatch.h 2024-08-08T20:48:50.0047454Z inflating: build/lib/torch/include/ATen/ops/unique_dim_consecutive_native.h 2024-08-08T20:48:50.0048506Z inflating: build/lib/torch/include/ATen/ops/unique_dim_consecutive_ops.h 2024-08-08T20:48:50.0049413Z inflating: build/lib/torch/include/ATen/ops/unique_dim_cpu_dispatch.h 2024-08-08T20:48:50.0050503Z inflating: build/lib/torch/include/ATen/ops/unique_dim_cuda_dispatch.h 2024-08-08T20:48:50.0051288Z inflating: build/lib/torch/include/ATen/ops/unique_dim_native.h 2024-08-08T20:48:50.0052423Z inflating: build/lib/torch/include/ATen/ops/unique_dim_ops.h 2024-08-08T20:48:50.0053267Z inflating: build/lib/torch/include/ATen/ops/unsafe_chunk.h 2024-08-08T20:48:50.0054677Z inflating: build/lib/torch/include/ATen/ops/unsafe_chunk_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0055279Z inflating: build/lib/torch/include/ATen/ops/unsafe_chunk_native.h 2024-08-08T20:48:50.0056375Z inflating: build/lib/torch/include/ATen/ops/unsafe_chunk_ops.h 2024-08-08T20:48:50.0057456Z inflating: build/lib/torch/include/ATen/ops/unsafe_split.h 2024-08-08T20:48:50.0058977Z inflating: build/lib/torch/include/ATen/ops/unsafe_split_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0059718Z inflating: build/lib/torch/include/ATen/ops/unsafe_split_native.h 2024-08-08T20:48:50.0060795Z inflating: build/lib/torch/include/ATen/ops/unsafe_split_ops.h 2024-08-08T20:48:50.0062579Z inflating: build/lib/torch/include/ATen/ops/unsafe_split_with_sizes.h 2024-08-08T20:48:50.0063565Z inflating: build/lib/torch/include/ATen/ops/unsafe_split_with_sizes_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0064201Z inflating: build/lib/torch/include/ATen/ops/unsafe_split_with_sizes_native.h 2024-08-08T20:48:50.0065068Z inflating: build/lib/torch/include/ATen/ops/unsafe_split_with_sizes_ops.h 2024-08-08T20:48:50.0065947Z inflating: build/lib/torch/include/ATen/ops/unsqueeze.h 2024-08-08T20:48:50.0067324Z inflating: build/lib/torch/include/ATen/ops/unsqueeze_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0068220Z inflating: build/lib/torch/include/ATen/ops/unsqueeze_copy.h 2024-08-08T20:48:50.0069974Z inflating: build/lib/torch/include/ATen/ops/unsqueeze_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0071191Z inflating: build/lib/torch/include/ATen/ops/unsqueeze_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0071847Z inflating: build/lib/torch/include/ATen/ops/unsqueeze_copy_native.h 2024-08-08T20:48:50.0072416Z inflating: build/lib/torch/include/ATen/ops/unsqueeze_copy_ops.h 2024-08-08T20:48:50.0072995Z inflating: build/lib/torch/include/ATen/ops/unsqueeze_native.h 2024-08-08T20:48:50.0074579Z inflating: build/lib/torch/include/ATen/ops/unsqueeze_ops.h 2024-08-08T20:48:50.0075613Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d.h 2024-08-08T20:48:50.0077274Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_backward.h 2024-08-08T20:48:50.0078473Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0079318Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_backward_cpu_dispatch.h 2024-08-08T20:48:50.0080439Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_backward_cuda_dispatch.h 2024-08-08T20:48:50.0081503Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_backward_meta.h 2024-08-08T20:48:50.0082971Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_backward_meta_dispatch.h 2024-08-08T20:48:50.0083647Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_backward_native.h 2024-08-08T20:48:50.0084557Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_backward_ops.h 2024-08-08T20:48:50.0085789Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0086684Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0087587Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_cpu_dispatch.h 2024-08-08T20:48:50.0088623Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_cuda_dispatch.h 2024-08-08T20:48:50.0089642Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_meta.h 2024-08-08T20:48:50.0090677Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_meta_dispatch.h 2024-08-08T20:48:50.0091631Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_native.h 2024-08-08T20:48:50.0092757Z inflating: build/lib/torch/include/ATen/ops/upsample_bicubic2d_ops.h 2024-08-08T20:48:50.0094443Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d.h 2024-08-08T20:48:50.0095510Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_backward.h 2024-08-08T20:48:50.0096877Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0097695Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_backward_cpu_dispatch.h 2024-08-08T20:48:50.0098798Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_backward_cuda_dispatch.h 2024-08-08T20:48:50.0099703Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_backward_meta.h 2024-08-08T20:48:50.0100841Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_backward_meta_dispatch.h 2024-08-08T20:48:50.0101907Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_backward_native.h 2024-08-08T20:48:50.0102963Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_backward_ops.h 2024-08-08T20:48:50.0104220Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0105105Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0105993Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_cpu_dispatch.h 2024-08-08T20:48:50.0107018Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_cuda_dispatch.h 2024-08-08T20:48:50.0107944Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_meta.h 2024-08-08T20:48:50.0109216Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_meta_dispatch.h 2024-08-08T20:48:50.0111076Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_native.h 2024-08-08T20:48:50.0111815Z inflating: build/lib/torch/include/ATen/ops/upsample_bilinear2d_ops.h 2024-08-08T20:48:50.0113367Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d.h 2024-08-08T20:48:50.0114424Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_backward.h 2024-08-08T20:48:50.0115858Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0116621Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_backward_cpu_dispatch.h 2024-08-08T20:48:50.0117702Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_backward_cuda_dispatch.h 2024-08-08T20:48:50.0118586Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_backward_meta.h 2024-08-08T20:48:50.0119771Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_backward_meta_dispatch.h 2024-08-08T20:48:50.0120741Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_backward_native.h 2024-08-08T20:48:50.0121799Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_backward_ops.h 2024-08-08T20:48:50.0123089Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0124028Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0124895Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_cpu_dispatch.h 2024-08-08T20:48:50.0125917Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_cuda_dispatch.h 2024-08-08T20:48:50.0126833Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_meta.h 2024-08-08T20:48:50.0127975Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_meta_dispatch.h 2024-08-08T20:48:50.0128950Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_native.h 2024-08-08T20:48:50.0130095Z inflating: build/lib/torch/include/ATen/ops/upsample_linear1d_ops.h 2024-08-08T20:48:50.0131668Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d.h 2024-08-08T20:48:50.0132639Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_backward.h 2024-08-08T20:48:50.0134040Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0134781Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_backward_cpu_dispatch.h 2024-08-08T20:48:50.0135840Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_backward_cuda_dispatch.h 2024-08-08T20:48:50.0136751Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_backward_meta.h 2024-08-08T20:48:50.0137873Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_backward_meta_dispatch.h 2024-08-08T20:48:50.0138806Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_backward_native.h 2024-08-08T20:48:50.0140039Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_backward_ops.h 2024-08-08T20:48:50.0141256Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0142134Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0143003Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_cpu_dispatch.h 2024-08-08T20:48:50.0144203Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_cuda_dispatch.h 2024-08-08T20:48:50.0145110Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_meta.h 2024-08-08T20:48:50.0146188Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_meta_dispatch.h 2024-08-08T20:48:50.0147144Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_native.h 2024-08-08T20:48:50.0148235Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest1d_ops.h 2024-08-08T20:48:50.0149841Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d.h 2024-08-08T20:48:50.0150829Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_backward.h 2024-08-08T20:48:50.0152234Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0153050Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_backward_cpu_dispatch.h 2024-08-08T20:48:50.0154170Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_backward_cuda_dispatch.h 2024-08-08T20:48:50.0155083Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_backward_meta.h 2024-08-08T20:48:50.0156238Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_backward_meta_dispatch.h 2024-08-08T20:48:50.0157158Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_backward_native.h 2024-08-08T20:48:50.0158259Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_backward_ops.h 2024-08-08T20:48:50.0159856Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0160750Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0161671Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_cpu_dispatch.h 2024-08-08T20:48:50.0162682Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_cuda_dispatch.h 2024-08-08T20:48:50.0163587Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_meta.h 2024-08-08T20:48:50.0164665Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_meta_dispatch.h 2024-08-08T20:48:50.0165640Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_native.h 2024-08-08T20:48:50.0166846Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest2d_ops.h 2024-08-08T20:48:50.0168460Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d.h 2024-08-08T20:48:50.0169533Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_backward.h 2024-08-08T20:48:50.0170968Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0171806Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_backward_cpu_dispatch.h 2024-08-08T20:48:50.0172881Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_backward_cuda_dispatch.h 2024-08-08T20:48:50.0173808Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_backward_meta.h 2024-08-08T20:48:50.0174958Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_backward_meta_dispatch.h 2024-08-08T20:48:50.0175871Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_backward_native.h 2024-08-08T20:48:50.0177015Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_backward_ops.h 2024-08-08T20:48:50.0178497Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0179485Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0181190Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_cpu_dispatch.h 2024-08-08T20:48:50.0181722Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_cuda_dispatch.h 2024-08-08T20:48:50.0182825Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_meta.h 2024-08-08T20:48:50.0183959Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_meta_dispatch.h 2024-08-08T20:48:50.0184915Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_native.h 2024-08-08T20:48:50.0186047Z inflating: build/lib/torch/include/ATen/ops/upsample_nearest3d_ops.h 2024-08-08T20:48:50.0187753Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d.h 2024-08-08T20:48:50.0189053Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_backward.h 2024-08-08T20:48:50.0190487Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_backward_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0191296Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_backward_cpu_dispatch.h 2024-08-08T20:48:50.0192417Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_backward_cuda_dispatch.h 2024-08-08T20:48:50.0193323Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_backward_meta.h 2024-08-08T20:48:50.0194494Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_backward_meta_dispatch.h 2024-08-08T20:48:50.0195443Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_backward_native.h 2024-08-08T20:48:50.0196568Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_backward_ops.h 2024-08-08T20:48:50.0197942Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0198910Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0199837Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_cpu_dispatch.h 2024-08-08T20:48:50.0200899Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_cuda_dispatch.h 2024-08-08T20:48:50.0201821Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_meta.h 2024-08-08T20:48:50.0202924Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_meta_dispatch.h 2024-08-08T20:48:50.0203894Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_native.h 2024-08-08T20:48:50.0205085Z inflating: build/lib/torch/include/ATen/ops/upsample_trilinear3d_ops.h 2024-08-08T20:48:50.0206224Z inflating: build/lib/torch/include/ATen/ops/value_selecting_reduction_backward.h 2024-08-08T20:48:50.0207506Z inflating: build/lib/torch/include/ATen/ops/value_selecting_reduction_backward_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0208221Z inflating: build/lib/torch/include/ATen/ops/value_selecting_reduction_backward_native.h 2024-08-08T20:48:50.0209248Z inflating: build/lib/torch/include/ATen/ops/value_selecting_reduction_backward_ops.h 2024-08-08T20:48:50.0209936Z inflating: build/lib/torch/include/ATen/ops/values.h 2024-08-08T20:48:50.0211197Z inflating: build/lib/torch/include/ATen/ops/values_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0211947Z inflating: build/lib/torch/include/ATen/ops/values_copy.h 2024-08-08T20:48:50.0213207Z inflating: build/lib/torch/include/ATen/ops/values_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0214519Z inflating: build/lib/torch/include/ATen/ops/values_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0215202Z inflating: build/lib/torch/include/ATen/ops/values_copy_native.h 2024-08-08T20:48:50.0216297Z inflating: build/lib/torch/include/ATen/ops/values_copy_ops.h 2024-08-08T20:48:50.0217186Z inflating: build/lib/torch/include/ATen/ops/values_native.h 2024-08-08T20:48:50.0218111Z inflating: build/lib/torch/include/ATen/ops/values_ops.h 2024-08-08T20:48:50.0219099Z inflating: build/lib/torch/include/ATen/ops/vander.h 2024-08-08T20:48:50.0220377Z inflating: build/lib/torch/include/ATen/ops/vander_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0221082Z inflating: build/lib/torch/include/ATen/ops/vander_native.h 2024-08-08T20:48:50.0222075Z inflating: build/lib/torch/include/ATen/ops/vander_ops.h 2024-08-08T20:48:50.0223543Z inflating: build/lib/torch/include/ATen/ops/var.h 2024-08-08T20:48:50.0224554Z inflating: build/lib/torch/include/ATen/ops/var_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0225376Z inflating: build/lib/torch/include/ATen/ops/var_cpu_dispatch.h 2024-08-08T20:48:50.0226355Z inflating: build/lib/torch/include/ATen/ops/var_cuda_dispatch.h 2024-08-08T20:48:50.0227378Z inflating: build/lib/torch/include/ATen/ops/var_mean.h 2024-08-08T20:48:50.0228739Z inflating: build/lib/torch/include/ATen/ops/var_mean_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0229792Z inflating: build/lib/torch/include/ATen/ops/var_mean_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0230588Z inflating: build/lib/torch/include/ATen/ops/var_mean_cpu_dispatch.h 2024-08-08T20:48:50.0231618Z inflating: build/lib/torch/include/ATen/ops/var_mean_cuda_dispatch.h 2024-08-08T20:48:50.0232575Z inflating: build/lib/torch/include/ATen/ops/var_mean_native.h 2024-08-08T20:48:50.0234162Z inflating: build/lib/torch/include/ATen/ops/var_mean_ops.h 2024-08-08T20:48:50.0234763Z inflating: build/lib/torch/include/ATen/ops/var_native.h 2024-08-08T20:48:50.0236457Z inflating: build/lib/torch/include/ATen/ops/var_ops.h 2024-08-08T20:48:50.0236907Z inflating: build/lib/torch/include/ATen/ops/vdot.h 2024-08-08T20:48:50.0238635Z inflating: build/lib/torch/include/ATen/ops/vdot_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0239118Z inflating: build/lib/torch/include/ATen/ops/vdot_cpu_dispatch.h 2024-08-08T20:48:50.0240109Z inflating: build/lib/torch/include/ATen/ops/vdot_cuda_dispatch.h 2024-08-08T20:48:50.0240990Z inflating: build/lib/torch/include/ATen/ops/vdot_native.h 2024-08-08T20:48:50.0242014Z inflating: build/lib/torch/include/ATen/ops/vdot_ops.h 2024-08-08T20:48:50.0242991Z inflating: build/lib/torch/include/ATen/ops/view.h 2024-08-08T20:48:50.0243986Z inflating: build/lib/torch/include/ATen/ops/view_as.h 2024-08-08T20:48:50.0244981Z inflating: build/lib/torch/include/ATen/ops/view_as_complex.h 2024-08-08T20:48:50.0246023Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_copy.h 2024-08-08T20:48:50.0247552Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0249551Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0250010Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_copy_native.h 2024-08-08T20:48:50.0250826Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_copy_ops.h 2024-08-08T20:48:50.0251904Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_cpu_dispatch.h 2024-08-08T20:48:50.0252867Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_cuda_dispatch.h 2024-08-08T20:48:50.0253871Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_meta_dispatch.h 2024-08-08T20:48:50.0254756Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_native.h 2024-08-08T20:48:50.0255735Z inflating: build/lib/torch/include/ATen/ops/view_as_complex_ops.h 2024-08-08T20:48:50.0256951Z inflating: build/lib/torch/include/ATen/ops/view_as_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0257743Z inflating: build/lib/torch/include/ATen/ops/view_as_native.h 2024-08-08T20:48:50.0258907Z inflating: build/lib/torch/include/ATen/ops/view_as_ops.h 2024-08-08T20:48:50.0259936Z inflating: build/lib/torch/include/ATen/ops/view_as_real.h 2024-08-08T20:48:50.0260953Z inflating: build/lib/torch/include/ATen/ops/view_as_real_copy.h 2024-08-08T20:48:50.0262326Z inflating: build/lib/torch/include/ATen/ops/view_as_real_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0263388Z inflating: build/lib/torch/include/ATen/ops/view_as_real_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0264060Z inflating: build/lib/torch/include/ATen/ops/view_as_real_copy_native.h 2024-08-08T20:48:50.0265073Z inflating: build/lib/torch/include/ATen/ops/view_as_real_copy_ops.h 2024-08-08T20:48:50.0266082Z inflating: build/lib/torch/include/ATen/ops/view_as_real_cpu_dispatch.h 2024-08-08T20:48:50.0267065Z inflating: build/lib/torch/include/ATen/ops/view_as_real_cuda_dispatch.h 2024-08-08T20:48:50.0268011Z inflating: build/lib/torch/include/ATen/ops/view_as_real_meta_dispatch.h 2024-08-08T20:48:50.0268924Z inflating: build/lib/torch/include/ATen/ops/view_as_real_native.h 2024-08-08T20:48:50.0269875Z inflating: build/lib/torch/include/ATen/ops/view_as_real_ops.h 2024-08-08T20:48:50.0271060Z inflating: build/lib/torch/include/ATen/ops/view_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0272042Z inflating: build/lib/torch/include/ATen/ops/view_copy.h 2024-08-08T20:48:50.0273366Z inflating: build/lib/torch/include/ATen/ops/view_copy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0274496Z inflating: build/lib/torch/include/ATen/ops/view_copy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0275099Z inflating: build/lib/torch/include/ATen/ops/view_copy_native.h 2024-08-08T20:48:50.0276204Z inflating: build/lib/torch/include/ATen/ops/view_copy_ops.h 2024-08-08T20:48:50.0277207Z inflating: build/lib/torch/include/ATen/ops/view_cpu_dispatch.h 2024-08-08T20:48:50.0278249Z inflating: build/lib/torch/include/ATen/ops/view_cuda_dispatch.h 2024-08-08T20:48:50.0279230Z inflating: build/lib/torch/include/ATen/ops/view_meta_dispatch.h 2024-08-08T20:48:50.0280313Z inflating: build/lib/torch/include/ATen/ops/view_native.h 2024-08-08T20:48:50.0281372Z inflating: build/lib/torch/include/ATen/ops/view_ops.h 2024-08-08T20:48:50.0282336Z inflating: build/lib/torch/include/ATen/ops/vsplit.h 2024-08-08T20:48:50.0283626Z inflating: build/lib/torch/include/ATen/ops/vsplit_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0284370Z inflating: build/lib/torch/include/ATen/ops/vsplit_native.h 2024-08-08T20:48:50.0285379Z inflating: build/lib/torch/include/ATen/ops/vsplit_ops.h 2024-08-08T20:48:50.0286329Z inflating: build/lib/torch/include/ATen/ops/vstack.h 2024-08-08T20:48:50.0287671Z inflating: build/lib/torch/include/ATen/ops/vstack_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0288412Z inflating: build/lib/torch/include/ATen/ops/vstack_native.h 2024-08-08T20:48:50.0289425Z inflating: build/lib/torch/include/ATen/ops/vstack_ops.h 2024-08-08T20:48:50.0290408Z inflating: build/lib/torch/include/ATen/ops/where.h 2024-08-08T20:48:50.0291699Z inflating: build/lib/torch/include/ATen/ops/where_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0292595Z inflating: build/lib/torch/include/ATen/ops/where_cpu_dispatch.h 2024-08-08T20:48:50.0293617Z inflating: build/lib/torch/include/ATen/ops/where_cuda_dispatch.h 2024-08-08T20:48:50.0294581Z inflating: build/lib/torch/include/ATen/ops/where_native.h 2024-08-08T20:48:50.0296120Z inflating: build/lib/torch/include/ATen/ops/where_ops.h 2024-08-08T20:48:50.0296692Z inflating: build/lib/torch/include/ATen/ops/xlogy.h 2024-08-08T20:48:50.0298445Z inflating: build/lib/torch/include/ATen/ops/xlogy_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0299489Z inflating: build/lib/torch/include/ATen/ops/xlogy_compositeexplicitautogradnonfunctional_dispatch.h 2024-08-08T20:48:50.0300069Z inflating: build/lib/torch/include/ATen/ops/xlogy_cpu_dispatch.h 2024-08-08T20:48:50.0300943Z inflating: build/lib/torch/include/ATen/ops/xlogy_cuda_dispatch.h 2024-08-08T20:48:50.0301880Z inflating: build/lib/torch/include/ATen/ops/xlogy_meta.h 2024-08-08T20:48:50.0302948Z inflating: build/lib/torch/include/ATen/ops/xlogy_meta_dispatch.h 2024-08-08T20:48:50.0303900Z inflating: build/lib/torch/include/ATen/ops/xlogy_native.h 2024-08-08T20:48:50.0305592Z inflating: build/lib/torch/include/ATen/ops/xlogy_ops.h 2024-08-08T20:48:50.0306098Z inflating: build/lib/torch/include/ATen/ops/xor.h 2024-08-08T20:48:50.0307738Z inflating: build/lib/torch/include/ATen/ops/xor_compositeimplicitautograd_dispatch.h 2024-08-08T20:48:50.0308181Z inflating: build/lib/torch/include/ATen/ops/xor_native.h 2024-08-08T20:48:50.0309231Z inflating: build/lib/torch/include/ATen/ops/xor_ops.h 2024-08-08T20:48:50.0310227Z inflating: build/lib/torch/include/ATen/ops/zero.h 2024-08-08T20:48:50.0311511Z inflating: build/lib/torch/include/ATen/ops/zero_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0312313Z inflating: build/lib/torch/include/ATen/ops/zero_cpu_dispatch.h 2024-08-08T20:48:50.0313434Z inflating: build/lib/torch/include/ATen/ops/zero_cuda_dispatch.h 2024-08-08T20:48:50.0315322Z inflating: build/lib/torch/include/ATen/ops/zero_meta_dispatch.h 2024-08-08T20:48:50.0315771Z inflating: build/lib/torch/include/ATen/ops/zero_native.h 2024-08-08T20:48:50.0317243Z inflating: build/lib/torch/include/ATen/ops/zero_ops.h 2024-08-08T20:48:50.0318248Z inflating: build/lib/torch/include/ATen/ops/zeros.h 2024-08-08T20:48:50.0319581Z inflating: build/lib/torch/include/ATen/ops/zeros_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0320412Z inflating: build/lib/torch/include/ATen/ops/zeros_like.h 2024-08-08T20:48:50.0321812Z inflating: build/lib/torch/include/ATen/ops/zeros_like_compositeexplicitautograd_dispatch.h 2024-08-08T20:48:50.0322928Z inflating: build/lib/torch/include/ATen/ops/zeros_like_compositeimplicitautogradnestedtensor_dispatch.h 2024-08-08T20:48:50.0323643Z inflating: build/lib/torch/include/ATen/ops/zeros_like_native.h 2024-08-08T20:48:50.0324737Z inflating: build/lib/torch/include/ATen/ops/zeros_like_ops.h 2024-08-08T20:48:50.0325716Z inflating: build/lib/torch/include/ATen/ops/zeros_native.h 2024-08-08T20:48:50.0327217Z inflating: build/lib/torch/include/ATen/ops/zeros_ops.h 2024-08-08T20:48:50.0327442Z creating: build/lib/torch/include/ATen/hip/ 2024-08-08T20:48:50.0327727Z creating: build/lib/torch/include/ATen/hip/impl/ 2024-08-08T20:48:50.0329148Z inflating: build/lib/torch/include/ATen/hip/impl/HIPAllocatorMasqueradingAsCUDA.h 2024-08-08T20:48:50.0330020Z inflating: build/lib/torch/include/ATen/hip/impl/HIPCachingAllocatorMasqueradingAsCUDA.h 2024-08-08T20:48:50.0332455Z inflating: build/lib/torch/include/ATen/hip/impl/HIPGuardImplMasqueradingAsCUDA.h 2024-08-08T20:48:50.0333294Z inflating: build/lib/torch/include/ATen/hip/impl/HIPStreamMasqueradingAsCUDA.h 2024-08-08T20:48:50.0333514Z creating: build/lib/torch/include/ATen/mps/ 2024-08-08T20:48:50.0334491Z inflating: build/lib/torch/include/ATen/mps/EmptyTensor.h 2024-08-08T20:48:50.0337327Z inflating: build/lib/torch/include/ATen/mps/IndexKernels.h 2024-08-08T20:48:50.0339879Z inflating: build/lib/torch/include/ATen/mps/MPSAllocator.h 2024-08-08T20:48:50.0340685Z inflating: build/lib/torch/include/ATen/mps/MPSAllocatorInterface.h 2024-08-08T20:48:50.0341603Z inflating: build/lib/torch/include/ATen/mps/MPSDevice.h 2024-08-08T20:48:50.0343080Z inflating: build/lib/torch/include/ATen/mps/MPSEvent.h 2024-08-08T20:48:50.0343702Z inflating: build/lib/torch/include/ATen/mps/MPSGeneratorImpl.h 2024-08-08T20:48:50.0345261Z inflating: build/lib/torch/include/ATen/mps/MPSGuardImpl.h 2024-08-08T20:48:50.0346347Z inflating: build/lib/torch/include/ATen/mps/MPSHooks.h 2024-08-08T20:48:50.0348988Z inflating: build/lib/torch/include/ATen/mps/MPSProfiler.h 2024-08-08T20:48:50.0349906Z inflating: build/lib/torch/include/ATen/mps/MPSStream.h 2024-08-08T20:48:50.0350259Z creating: build/lib/torch/include/ATen/miopen/ 2024-08-08T20:48:50.0352285Z inflating: build/lib/torch/include/ATen/miopen/Descriptors.h 2024-08-08T20:48:50.0352745Z inflating: build/lib/torch/include/ATen/miopen/Exceptions.h 2024-08-08T20:48:50.0353573Z inflating: build/lib/torch/include/ATen/miopen/Handle.h 2024-08-08T20:48:50.0354409Z inflating: build/lib/torch/include/ATen/miopen/Types.h 2024-08-08T20:48:50.0355241Z inflating: build/lib/torch/include/ATen/miopen/Utils.h 2024-08-08T20:48:50.0356260Z inflating: build/lib/torch/include/ATen/miopen/miopen-wrapper.h 2024-08-08T20:48:50.0356527Z creating: build/lib/torch/include/ATen/detail/ 2024-08-08T20:48:50.0357688Z inflating: build/lib/torch/include/ATen/detail/AcceleratorHooksInterface.h 2024-08-08T20:48:50.0359661Z inflating: build/lib/torch/include/ATen/detail/CUDAHooksInterface.h 2024-08-08T20:48:50.0361359Z inflating: build/lib/torch/include/ATen/detail/FunctionTraits.h 2024-08-08T20:48:50.0361973Z inflating: build/lib/torch/include/ATen/detail/HIPHooksInterface.h 2024-08-08T20:48:50.0363064Z inflating: build/lib/torch/include/ATen/detail/IPUHooksInterface.h 2024-08-08T20:48:50.0364043Z inflating: build/lib/torch/include/ATen/detail/MAIAHooksInterface.h 2024-08-08T20:48:50.0365782Z inflating: build/lib/torch/include/ATen/detail/MPSHooksInterface.h 2024-08-08T20:48:50.0366560Z inflating: build/lib/torch/include/ATen/detail/MTIAHooksInterface.h 2024-08-08T20:48:50.0367719Z inflating: build/lib/torch/include/ATen/detail/PrivateUse1HooksInterface.h 2024-08-08T20:48:50.0368751Z inflating: build/lib/torch/include/ATen/detail/XPUHooksInterface.h 2024-08-08T20:48:50.0369216Z creating: build/lib/torch/include/ATen/native/ 2024-08-08T20:48:50.0370752Z inflating: build/lib/torch/include/ATen/native/Activation.h 2024-08-08T20:48:50.0371322Z inflating: build/lib/torch/include/ATen/native/AdaptivePooling.h 2024-08-08T20:48:50.0372262Z inflating: build/lib/torch/include/ATen/native/AmpKernels.h 2024-08-08T20:48:50.0374134Z inflating: build/lib/torch/include/ATen/native/BatchLinearAlgebra.h 2024-08-08T20:48:50.0374973Z inflating: build/lib/torch/include/ATen/native/BinaryOps.h 2024-08-08T20:48:50.0377237Z inflating: build/lib/torch/include/ATen/native/BucketizationUtils.h 2024-08-08T20:48:50.0377787Z inflating: build/lib/torch/include/ATen/native/CPUBlas.h 2024-08-08T20:48:50.0378861Z inflating: build/lib/torch/include/ATen/native/CPUFallback.h 2024-08-08T20:48:50.0380023Z inflating: build/lib/torch/include/ATen/native/CanUse32BitIndexMath.h 2024-08-08T20:48:50.0381282Z inflating: build/lib/torch/include/ATen/native/ComplexHelper.h 2024-08-08T20:48:50.0382413Z inflating: build/lib/torch/include/ATen/native/CompositeRandomAccessor.h 2024-08-08T20:48:50.0384188Z inflating: build/lib/torch/include/ATen/native/CompositeRandomAccessorCommon.h 2024-08-08T20:48:50.0386267Z inflating: build/lib/torch/include/ATen/native/ConvUtils.h 2024-08-08T20:48:50.0386727Z inflating: build/lib/torch/include/ATen/native/ConvolutionMM3d.h 2024-08-08T20:48:50.0387648Z inflating: build/lib/torch/include/ATen/native/Copy.h 2024-08-08T20:48:50.0388447Z inflating: build/lib/torch/include/ATen/native/Cross.h 2024-08-08T20:48:50.0390369Z inflating: build/lib/torch/include/ATen/native/DilatedConvolutionUtils.h 2024-08-08T20:48:50.0392234Z inflating: build/lib/torch/include/ATen/native/DispatchStub.h 2024-08-08T20:48:50.0392647Z inflating: build/lib/torch/include/ATen/native/Distance.h 2024-08-08T20:48:50.0395513Z inflating: build/lib/torch/include/ATen/native/DistributionTemplates.h 2024-08-08T20:48:50.0398267Z inflating: build/lib/torch/include/ATen/native/Distributions.h 2024-08-08T20:48:50.0399417Z inflating: build/lib/torch/include/ATen/native/EmbeddingBag.h 2024-08-08T20:48:50.0400089Z inflating: build/lib/torch/include/ATen/native/Fill.h 2024-08-08T20:48:50.0402434Z inflating: build/lib/torch/include/ATen/native/ForeachUtils.h 2024-08-08T20:48:50.0403101Z inflating: build/lib/torch/include/ATen/native/FractionalMaxPooling.h 2024-08-08T20:48:50.0404149Z inflating: build/lib/torch/include/ATen/native/FunctionOfAMatrixUtils.h 2024-08-08T20:48:50.0404933Z inflating: build/lib/torch/include/ATen/native/FusedAdagrad.h 2024-08-08T20:48:50.0405768Z inflating: build/lib/torch/include/ATen/native/FusedAdam.h 2024-08-08T20:48:50.0406672Z inflating: build/lib/torch/include/ATen/native/FusedSGD.h 2024-08-08T20:48:50.0408788Z inflating: build/lib/torch/include/ATen/native/GridSampler.h 2024-08-08T20:48:50.0409612Z inflating: build/lib/torch/include/ATen/native/GridSamplerUtils.h 2024-08-08T20:48:50.0410443Z inflating: build/lib/torch/include/ATen/native/Histogram.h 2024-08-08T20:48:50.0411435Z inflating: build/lib/torch/include/ATen/native/IndexKernel.h 2024-08-08T20:48:50.0413192Z inflating: build/lib/torch/include/ATen/native/IndexingUtils.h 2024-08-08T20:48:50.0413646Z inflating: build/lib/torch/include/ATen/native/Lerp.h 2024-08-08T20:48:50.0414703Z inflating: build/lib/torch/include/ATen/native/LinearAlgebra.h 2024-08-08T20:48:50.0418440Z inflating: build/lib/torch/include/ATen/native/LinearAlgebraUtils.h 2024-08-08T20:48:50.0418875Z inflating: build/lib/torch/include/ATen/native/LossMulti.h 2024-08-08T20:48:50.0432215Z inflating: build/lib/torch/include/ATen/native/Math.h 2024-08-08T20:48:50.0433195Z inflating: build/lib/torch/include/ATen/native/MathBitFallThroughLists.h 2024-08-08T20:48:50.0435028Z inflating: build/lib/torch/include/ATen/native/MathBitsFallback.h 2024-08-08T20:48:50.0435814Z inflating: build/lib/torch/include/ATen/native/MaxPooling.h 2024-08-08T20:48:50.0436885Z inflating: build/lib/torch/include/ATen/native/NonEmptyUtils.h 2024-08-08T20:48:50.0438031Z inflating: build/lib/torch/include/ATen/native/NonSymbolicBC.h 2024-08-08T20:48:50.0438850Z inflating: build/lib/torch/include/ATen/native/Normalization.h 2024-08-08T20:48:50.0439819Z inflating: build/lib/torch/include/ATen/native/Padding.h 2024-08-08T20:48:50.0440849Z inflating: build/lib/torch/include/ATen/native/PixelShuffle.h 2024-08-08T20:48:50.0441723Z inflating: build/lib/torch/include/ATen/native/PointwiseOps.h 2024-08-08T20:48:50.0443912Z inflating: build/lib/torch/include/ATen/native/Pool.h 2024-08-08T20:48:50.0444425Z inflating: build/lib/torch/include/ATen/native/Pow.h 2024-08-08T20:48:50.0446383Z inflating: build/lib/torch/include/ATen/native/RNN.h 2024-08-08T20:48:50.0446932Z inflating: build/lib/torch/include/ATen/native/RangeFactories.h 2024-08-08T20:48:50.0447915Z inflating: build/lib/torch/include/ATen/native/ReduceAllOps.h 2024-08-08T20:48:50.0448861Z inflating: build/lib/torch/include/ATen/native/ReduceOps.h 2024-08-08T20:48:50.0451416Z inflating: build/lib/torch/include/ATen/native/ReduceOpsUtils.h 2024-08-08T20:48:50.0451975Z inflating: build/lib/torch/include/ATen/native/ReductionType.h 2024-08-08T20:48:50.0452977Z inflating: build/lib/torch/include/ATen/native/Repeat.h 2024-08-08T20:48:50.0454726Z inflating: build/lib/torch/include/ATen/native/Resize.h 2024-08-08T20:48:50.0455384Z inflating: build/lib/torch/include/ATen/native/ResizeCommon.h 2024-08-08T20:48:50.0457094Z inflating: build/lib/torch/include/ATen/native/ScatterGatherChecks.h 2024-08-08T20:48:50.0457626Z inflating: build/lib/torch/include/ATen/native/SegmentReduce.h 2024-08-08T20:48:50.0460461Z inflating: build/lib/torch/include/ATen/native/SharedReduceOps.h 2024-08-08T20:48:50.0460996Z inflating: build/lib/torch/include/ATen/native/SobolEngineOpsUtils.h 2024-08-08T20:48:50.0461907Z inflating: build/lib/torch/include/ATen/native/Sorting.h 2024-08-08T20:48:50.0462964Z inflating: build/lib/torch/include/ATen/native/SortingUtils.h 2024-08-08T20:48:50.0464745Z inflating: build/lib/torch/include/ATen/native/SparseTensorUtils.h 2024-08-08T20:48:50.0466188Z inflating: build/lib/torch/include/ATen/native/SpectralOpsUtils.h 2024-08-08T20:48:50.0467428Z inflating: build/lib/torch/include/ATen/native/StridedRandomAccessor.h 2024-08-08T20:48:50.0468738Z inflating: build/lib/torch/include/ATen/native/TensorAdvancedIndexing.h 2024-08-08T20:48:50.0470187Z inflating: build/lib/torch/include/ATen/native/TensorAdvancedIndexingUtils.h 2024-08-08T20:48:50.0471254Z inflating: build/lib/torch/include/ATen/native/TensorCompare.h 2024-08-08T20:48:50.0472190Z inflating: build/lib/torch/include/ATen/native/TensorConversions.h 2024-08-08T20:48:50.0473145Z inflating: build/lib/torch/include/ATen/native/TensorDimApply.h 2024-08-08T20:48:50.0474282Z inflating: build/lib/torch/include/ATen/native/TensorFactories.h 2024-08-08T20:48:50.0475254Z extracting: build/lib/torch/include/ATen/native/TensorIterator.h 2024-08-08T20:48:50.0476475Z inflating: build/lib/torch/include/ATen/native/TensorIteratorDynamicCasting.h 2024-08-08T20:48:50.0477555Z inflating: build/lib/torch/include/ATen/native/TensorProperties.h 2024-08-08T20:48:50.0478980Z inflating: build/lib/torch/include/ATen/native/TensorShape.h 2024-08-08T20:48:50.0480636Z inflating: build/lib/torch/include/ATen/native/TensorTransformations.h 2024-08-08T20:48:50.0481962Z inflating: build/lib/torch/include/ATen/native/TopKImpl.h 2024-08-08T20:48:50.0483307Z inflating: build/lib/torch/include/ATen/native/TransposeType.h 2024-08-08T20:48:50.0484578Z inflating: build/lib/torch/include/ATen/native/TriangularOpsUtils.h 2024-08-08T20:48:50.0486121Z inflating: build/lib/torch/include/ATen/native/TypeProperties.h 2024-08-08T20:48:50.0487500Z inflating: build/lib/torch/include/ATen/native/UnaryOps.h 2024-08-08T20:48:50.0488792Z inflating: build/lib/torch/include/ATen/native/Unfold2d.h 2024-08-08T20:48:50.0489637Z inflating: build/lib/torch/include/ATen/native/Unfold3d.h 2024-08-08T20:48:50.0490545Z inflating: build/lib/torch/include/ATen/native/UnfoldBackward.h 2024-08-08T20:48:50.0491938Z inflating: build/lib/torch/include/ATen/native/UpSample.h 2024-08-08T20:48:50.0493254Z inflating: build/lib/torch/include/ATen/native/batch_norm.h 2024-08-08T20:48:50.0494535Z inflating: build/lib/torch/include/ATen/native/group_norm.h 2024-08-08T20:48:50.0495692Z inflating: build/lib/torch/include/ATen/native/im2col.h 2024-08-08T20:48:50.0496914Z inflating: build/lib/torch/include/ATen/native/im2col_shape_check.h 2024-08-08T20:48:50.0498291Z inflating: build/lib/torch/include/ATen/native/layer_norm.h 2024-08-08T20:48:50.0499478Z inflating: build/lib/torch/include/ATen/native/verbose_wrapper.h 2024-08-08T20:48:50.0500835Z inflating: build/lib/torch/include/ATen/native/vol2col.h 2024-08-08T20:48:50.0502482Z creating: build/lib/torch/include/ATen/native/cpu/ 2024-08-08T20:48:50.0503883Z inflating: build/lib/torch/include/ATen/native/cpu/AtomicAddFloat.h 2024-08-08T20:48:50.0504883Z inflating: build/lib/torch/include/ATen/native/cpu/CatKernel.h 2024-08-08T20:48:50.0506316Z inflating: build/lib/torch/include/ATen/native/cpu/ChannelShuffleKernel.h 2024-08-08T20:48:50.0507552Z inflating: build/lib/torch/include/ATen/native/cpu/CopyKernel.h 2024-08-08T20:48:50.0509204Z inflating: build/lib/torch/include/ATen/native/cpu/DepthwiseConvKernel.h 2024-08-08T20:48:50.0510783Z inflating: build/lib/torch/include/ATen/native/cpu/DistributionTemplates.h 2024-08-08T20:48:50.0512258Z inflating: build/lib/torch/include/ATen/native/cpu/GridSamplerKernel.h 2024-08-08T20:48:50.0513509Z inflating: build/lib/torch/include/ATen/native/cpu/IndexKernelUtils.h 2024-08-08T20:48:50.0515104Z inflating: build/lib/torch/include/ATen/native/cpu/Intrinsics.h 2024-08-08T20:48:50.0516262Z inflating: build/lib/torch/include/ATen/native/cpu/IsContiguous.h 2024-08-08T20:48:50.0517615Z inflating: build/lib/torch/include/ATen/native/cpu/LogAddExp.h 2024-08-08T20:48:50.0518764Z inflating: build/lib/torch/include/ATen/native/cpu/Loops.h 2024-08-08T20:48:50.0520145Z inflating: build/lib/torch/include/ATen/native/cpu/MaxUnpoolKernel.h 2024-08-08T20:48:50.0521578Z inflating: build/lib/torch/include/ATen/native/cpu/PixelShuffleKernel.h 2024-08-08T20:48:50.0523122Z inflating: build/lib/torch/include/ATen/native/cpu/Reduce.h 2024-08-08T20:48:50.0524478Z inflating: build/lib/torch/include/ATen/native/cpu/ReduceUtils.h 2024-08-08T20:48:50.0525489Z inflating: build/lib/torch/include/ATen/native/cpu/SampledAddmmKernel.h 2024-08-08T20:48:50.0526514Z inflating: build/lib/torch/include/ATen/native/cpu/SerialStackImpl.h 2024-08-08T20:48:50.0527508Z inflating: build/lib/torch/include/ATen/native/cpu/SoftmaxKernel.h 2024-08-08T20:48:50.0528529Z inflating: build/lib/torch/include/ATen/native/cpu/SpmmReduceKernel.h 2024-08-08T20:48:50.0530002Z inflating: build/lib/torch/include/ATen/native/cpu/StackKernel.h 2024-08-08T20:48:50.0531248Z inflating: build/lib/torch/include/ATen/native/cpu/UpSampleKernelAVXAntialias.h 2024-08-08T20:48:50.0532702Z inflating: build/lib/torch/include/ATen/native/cpu/WeightNormKernel.h 2024-08-08T20:48:50.0534205Z inflating: build/lib/torch/include/ATen/native/cpu/avx_mathfun.h 2024-08-08T20:48:50.0535607Z inflating: build/lib/torch/include/ATen/native/cpu/int_mm_kernel.h 2024-08-08T20:48:50.0536969Z inflating: build/lib/torch/include/ATen/native/cpu/mixed_data_type.h 2024-08-08T20:48:50.0538314Z inflating: build/lib/torch/include/ATen/native/cpu/moments_utils.h 2024-08-08T20:48:50.0539632Z inflating: build/lib/torch/include/ATen/native/cpu/utils.h 2024-08-08T20:48:50.0540973Z inflating: build/lib/torch/include/ATen/native/cpu/zmath.h 2024-08-08T20:48:50.0542229Z creating: build/lib/torch/include/ATen/native/cuda/ 2024-08-08T20:48:50.0543563Z inflating: build/lib/torch/include/ATen/native/cuda/Activation.h 2024-08-08T20:48:50.0544559Z inflating: build/lib/torch/include/ATen/native/cuda/BinaryInternal.h 2024-08-08T20:48:50.0545985Z inflating: build/lib/torch/include/ATen/native/cuda/CompositeRandomAccessor.h 2024-08-08T20:48:50.0547325Z inflating: build/lib/torch/include/ATen/native/cuda/Copy.h 2024-08-08T20:48:50.0548228Z inflating: build/lib/torch/include/ATen/native/cuda/CuFFTPlanCache.h 2024-08-08T20:48:50.0549191Z inflating: build/lib/torch/include/ATen/native/cuda/CuFFTUtils.h 2024-08-08T20:48:50.0550959Z inflating: build/lib/torch/include/ATen/native/cuda/DistributionTemplates.h 2024-08-08T20:48:50.0552624Z inflating: build/lib/torch/include/ATen/native/cuda/Distributions.h 2024-08-08T20:48:50.0554150Z inflating: build/lib/torch/include/ATen/native/cuda/GridSampler.h 2024-08-08T20:48:50.0555678Z inflating: build/lib/torch/include/ATen/native/cuda/IndexKernel.h 2024-08-08T20:48:50.0557394Z inflating: build/lib/torch/include/ATen/native/cuda/LaunchUtils.h 2024-08-08T20:48:50.0559103Z inflating: build/lib/torch/include/ATen/native/cuda/MiscUtils.h 2024-08-08T20:48:50.0560591Z inflating: build/lib/torch/include/ATen/native/cuda/ReduceOps.h 2024-08-08T20:48:50.0562007Z inflating: build/lib/torch/include/ATen/native/cuda/Resize.h 2024-08-08T20:48:50.0563464Z inflating: build/lib/torch/include/ATen/native/cuda/RowwiseScaledMM.h 2024-08-08T20:48:50.0564821Z inflating: build/lib/torch/include/ATen/native/cuda/ScanKernels.h 2024-08-08T20:48:50.0566078Z inflating: build/lib/torch/include/ATen/native/cuda/Sort.h 2024-08-08T20:48:50.0567067Z inflating: build/lib/torch/include/ATen/native/cuda/SortStable.h 2024-08-08T20:48:50.0568401Z inflating: build/lib/torch/include/ATen/native/cuda/Sorting.h 2024-08-08T20:48:50.0569588Z inflating: build/lib/torch/include/ATen/native/cuda/TensorModeKernel.h 2024-08-08T20:48:50.0570901Z inflating: build/lib/torch/include/ATen/native/cuda/TensorTopK.h 2024-08-08T20:48:50.0572239Z inflating: build/lib/torch/include/ATen/native/cuda/jit_utils.h 2024-08-08T20:48:50.0573453Z inflating: build/lib/torch/include/ATen/native/cuda/thread_constants.h 2024-08-08T20:48:50.0575099Z inflating: build/lib/torch/include/ATen/native/cuda/CUDAJitLoops.cuh 2024-08-08T20:48:50.0576235Z inflating: build/lib/torch/include/ATen/native/cuda/CUDALoops.cuh 2024-08-08T20:48:50.0577569Z inflating: build/lib/torch/include/ATen/native/cuda/DeviceSqrt.cuh 2024-08-08T20:48:50.0579036Z inflating: build/lib/torch/include/ATen/native/cuda/EmbeddingBackwardKernel.cuh 2024-08-08T20:48:50.0580576Z inflating: build/lib/torch/include/ATen/native/cuda/ForeachFunctors.cuh 2024-08-08T20:48:50.0582132Z inflating: build/lib/torch/include/ATen/native/cuda/ForeachMinMaxFunctors.cuh 2024-08-08T20:48:50.0583242Z inflating: build/lib/torch/include/ATen/native/cuda/GridSampler.cuh 2024-08-08T20:48:50.0584208Z inflating: build/lib/torch/include/ATen/native/cuda/JitLoops.cuh 2024-08-08T20:48:50.0585141Z inflating: build/lib/torch/include/ATen/native/cuda/KernelUtils.cuh 2024-08-08T20:48:50.0586080Z inflating: build/lib/torch/include/ATen/native/cuda/Loops.cuh 2024-08-08T20:48:50.0592958Z inflating: build/lib/torch/include/ATen/native/cuda/Math.cuh 2024-08-08T20:48:50.0594996Z inflating: build/lib/torch/include/ATen/native/cuda/MemoryAccess.cuh 2024-08-08T20:48:50.0596968Z inflating: build/lib/torch/include/ATen/native/cuda/MultiTensorApply.cuh 2024-08-08T20:48:50.0603142Z inflating: build/lib/torch/include/ATen/native/cuda/Normalization.cuh 2024-08-08T20:48:50.0605385Z inflating: build/lib/torch/include/ATen/native/cuda/PersistentSoftmax.cuh 2024-08-08T20:48:50.0607140Z inflating: build/lib/torch/include/ATen/native/cuda/Pow.cuh 2024-08-08T20:48:50.0608115Z inflating: build/lib/torch/include/ATen/native/cuda/Randperm.cuh 2024-08-08T20:48:50.0612468Z inflating: build/lib/torch/include/ATen/native/cuda/Reduce.cuh 2024-08-08T20:48:50.0614978Z inflating: build/lib/torch/include/ATen/native/cuda/ScanUtils.cuh 2024-08-08T20:48:50.0616761Z inflating: build/lib/torch/include/ATen/native/cuda/SortUtils.cuh 2024-08-08T20:48:50.0618146Z inflating: build/lib/torch/include/ATen/native/cuda/SortingCommon.cuh 2024-08-08T20:48:50.0620016Z inflating: build/lib/torch/include/ATen/native/cuda/SortingRadixSelect.cuh 2024-08-08T20:48:50.0622611Z inflating: build/lib/torch/include/ATen/native/cuda/TensorModeKernel.cuh 2024-08-08T20:48:50.0623644Z inflating: build/lib/torch/include/ATen/native/cuda/UniqueCub.cuh 2024-08-08T20:48:50.0624953Z inflating: build/lib/torch/include/ATen/native/cuda/UpSample.cuh 2024-08-08T20:48:50.0626204Z inflating: build/lib/torch/include/ATen/native/cuda/block_reduce.cuh 2024-08-08T20:48:50.0627773Z inflating: build/lib/torch/include/ATen/native/cuda/fused_adam_amsgrad_impl.cuh 2024-08-08T20:48:50.0629587Z inflating: build/lib/torch/include/ATen/native/cuda/fused_adam_impl.cuh 2024-08-08T20:48:50.0631215Z inflating: build/lib/torch/include/ATen/native/cuda/fused_adam_utils.cuh 2024-08-08T20:48:50.0632665Z inflating: build/lib/torch/include/ATen/native/cuda/fused_adamw_amsgrad_impl.cuh 2024-08-08T20:48:50.0634074Z inflating: build/lib/torch/include/ATen/native/cuda/fused_adamw_impl.cuh 2024-08-08T20:48:50.0635051Z inflating: build/lib/torch/include/ATen/native/cuda/im2col.cuh 2024-08-08T20:48:50.0636428Z inflating: build/lib/torch/include/ATen/native/cuda/reduction_template.cuh 2024-08-08T20:48:50.0637916Z inflating: build/lib/torch/include/ATen/native/cuda/vol2col.cuh 2024-08-08T20:48:50.0639211Z creating: build/lib/torch/include/ATen/native/mps/ 2024-08-08T20:48:50.0640248Z inflating: build/lib/torch/include/ATen/native/mps/Copy.h 2024-08-08T20:48:50.0641509Z inflating: build/lib/torch/include/ATen/native/mps/MPSGraphSonomaOps.h 2024-08-08T20:48:50.0642612Z inflating: build/lib/torch/include/ATen/native/mps/MPSGraphVenturaOps.h 2024-08-08T20:48:50.0644235Z inflating: build/lib/torch/include/ATen/native/mps/OperationUtils.h 2024-08-08T20:48:50.0645893Z inflating: build/lib/torch/include/ATen/native/mps/TensorFactory.h 2024-08-08T20:48:50.0647538Z inflating: build/lib/torch/include/ATen/native/mps/UnaryConstants.h 2024-08-08T20:48:50.0648793Z creating: build/lib/torch/include/ATen/native/nested/ 2024-08-08T20:48:50.0650247Z inflating: build/lib/torch/include/ATen/native/nested/NestedTensorBinaryOps.h 2024-08-08T20:48:50.0651735Z inflating: build/lib/torch/include/ATen/native/nested/NestedTensorFactories.h 2024-08-08T20:48:50.0652972Z inflating: build/lib/torch/include/ATen/native/nested/NestedTensorMath.h 2024-08-08T20:48:50.0654865Z inflating: build/lib/torch/include/ATen/native/nested/NestedTensorTransformerFunctions.h 2024-08-08T20:48:50.0667283Z inflating: build/lib/torch/include/ATen/native/nested/NestedTensorTransformerUtils.h 2024-08-08T20:48:50.0668922Z inflating: build/lib/torch/include/ATen/native/nested/NestedTensorUtils.h 2024-08-08T20:48:50.0670197Z creating: build/lib/torch/include/ATen/native/quantized/ 2024-08-08T20:48:50.0671558Z inflating: build/lib/torch/include/ATen/native/quantized/AffineQuantizer.h 2024-08-08T20:48:50.0673076Z inflating: build/lib/torch/include/ATen/native/quantized/AffineQuantizerBase.h 2024-08-08T20:48:50.0674372Z inflating: build/lib/torch/include/ATen/native/quantized/ConvUtils.h 2024-08-08T20:48:50.0675595Z inflating: build/lib/torch/include/ATen/native/quantized/Copy.h 2024-08-08T20:48:50.0676744Z inflating: build/lib/torch/include/ATen/native/quantized/FakeQuantAffine.h 2024-08-08T20:48:50.0678410Z inflating: build/lib/torch/include/ATen/native/quantized/IndexKernel.h 2024-08-08T20:48:50.0680033Z inflating: build/lib/torch/include/ATen/native/quantized/PackedParams.h 2024-08-08T20:48:50.0681502Z creating: build/lib/torch/include/ATen/native/quantized/cpu/ 2024-08-08T20:48:50.0682656Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/BinaryOps.h 2024-08-08T20:48:50.0684216Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/EmbeddingPackedParams.h 2024-08-08T20:48:50.0686121Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/OnednnUtils.h 2024-08-08T20:48:50.0687767Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/QnnpackUtils.h 2024-08-08T20:48:50.0689425Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/QuantUtils.h 2024-08-08T20:48:50.0690897Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/QuantizedOps.h 2024-08-08T20:48:50.0692640Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/RuyUtils.h 2024-08-08T20:48:50.0694429Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/XnnpackUtils.h 2024-08-08T20:48:50.0695991Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/conv_serialization.h 2024-08-08T20:48:50.0697724Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/fbgemm_utils.h 2024-08-08T20:48:50.0699703Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/init_qnnpack.h 2024-08-08T20:48:50.0700845Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/qembeddingbag.h 2024-08-08T20:48:50.0702069Z inflating: build/lib/torch/include/ATen/native/quantized/cpu/qembeddingbag_prepack.h 2024-08-08T20:48:50.0703130Z creating: build/lib/torch/include/ATen/native/transformers/ 2024-08-08T20:48:50.0704057Z inflating: build/lib/torch/include/ATen/native/transformers/attention.h 2024-08-08T20:48:50.0705141Z inflating: build/lib/torch/include/ATen/native/transformers/sdp_utils_cpp.h 2024-08-08T20:48:50.0706054Z creating: build/lib/torch/include/ATen/native/utils/ 2024-08-08T20:48:50.0706849Z inflating: build/lib/torch/include/ATen/native/utils/Factory.h 2024-08-08T20:48:50.0708069Z inflating: build/lib/torch/include/ATen/native/utils/ParamUtils.h 2024-08-08T20:48:50.0709596Z inflating: build/lib/torch/include/ATen/native/utils/ParamsHash.h 2024-08-08T20:48:50.0710606Z creating: build/lib/torch/include/ATen/quantized/ 2024-08-08T20:48:50.0711763Z inflating: build/lib/torch/include/ATen/quantized/QTensorImpl.h 2024-08-08T20:48:50.0713246Z inflating: build/lib/torch/include/ATen/quantized/Quantizer.h 2024-08-08T20:48:50.0714301Z creating: build/lib/torch/include/ATen/xpu/ 2024-08-08T20:48:50.0715435Z inflating: build/lib/torch/include/ATen/xpu/CachingHostAllocator.h 2024-08-08T20:48:50.0716750Z inflating: build/lib/torch/include/ATen/xpu/PinnedMemoryAllocator.h 2024-08-08T20:48:50.0718038Z inflating: build/lib/torch/include/ATen/xpu/XPUContext.h 2024-08-08T20:48:50.0718845Z inflating: build/lib/torch/include/ATen/xpu/XPUDevice.h 2024-08-08T20:48:50.0719614Z inflating: build/lib/torch/include/ATen/xpu/XPUEvent.h 2024-08-08T20:48:50.0720982Z inflating: build/lib/torch/include/ATen/xpu/XPUGeneratorImpl.h 2024-08-08T20:48:50.0722081Z creating: build/lib/torch/include/ATen/xpu/detail/ 2024-08-08T20:48:50.0722877Z inflating: build/lib/torch/include/ATen/xpu/detail/XPUHooks.h 2024-08-08T20:48:50.0723870Z creating: build/lib/torch/include/caffe2/ 2024-08-08T20:48:50.0724791Z creating: build/lib/torch/include/caffe2/serialize/ 2024-08-08T20:48:50.0725840Z inflating: build/lib/torch/include/caffe2/serialize/crc_alt.h 2024-08-08T20:48:50.0727124Z inflating: build/lib/torch/include/caffe2/serialize/file_adapter.h 2024-08-08T20:48:50.0728718Z inflating: build/lib/torch/include/caffe2/serialize/in_memory_adapter.h 2024-08-08T20:48:50.0730254Z inflating: build/lib/torch/include/caffe2/serialize/inline_container.h 2024-08-08T20:48:50.0731830Z inflating: build/lib/torch/include/caffe2/serialize/istream_adapter.h 2024-08-08T20:48:50.0733495Z inflating: build/lib/torch/include/caffe2/serialize/read_adapter_interface.h 2024-08-08T20:48:50.0734803Z inflating: build/lib/torch/include/caffe2/serialize/versions.h 2024-08-08T20:48:50.0736072Z creating: build/lib/torch/include/c10/ 2024-08-08T20:48:50.0736857Z creating: build/lib/torch/include/c10/macros/ 2024-08-08T20:48:50.0737568Z inflating: build/lib/torch/include/c10/macros/Export.h 2024-08-08T20:48:50.0738698Z inflating: build/lib/torch/include/c10/macros/Macros.h 2024-08-08T20:48:50.0739869Z inflating: build/lib/torch/include/c10/macros/cmake_macros.h 2024-08-08T20:48:50.0740625Z creating: build/lib/torch/include/c10/core/ 2024-08-08T20:48:50.0741297Z inflating: build/lib/torch/include/c10/core/Allocator.h 2024-08-08T20:48:50.0742109Z inflating: build/lib/torch/include/c10/core/AutogradState.h 2024-08-08T20:48:50.0743002Z inflating: build/lib/torch/include/c10/core/Backend.h 2024-08-08T20:48:50.0744078Z inflating: build/lib/torch/include/c10/core/CPUAllocator.h 2024-08-08T20:48:50.0745046Z inflating: build/lib/torch/include/c10/core/CompileTimeFunctionPointer.h 2024-08-08T20:48:50.0746082Z inflating: build/lib/torch/include/c10/core/ConstantSymNodeImpl.h 2024-08-08T20:48:50.0747088Z inflating: build/lib/torch/include/c10/core/Contiguity.h 2024-08-08T20:48:50.0748162Z inflating: build/lib/torch/include/c10/core/CopyBytes.h 2024-08-08T20:48:50.0749296Z inflating: build/lib/torch/include/c10/core/DefaultDtype.h 2024-08-08T20:48:50.0750550Z inflating: build/lib/torch/include/c10/core/DefaultTensorOptions.h 2024-08-08T20:48:50.0751758Z inflating: build/lib/torch/include/c10/core/Device.h 2024-08-08T20:48:50.0752973Z inflating: build/lib/torch/include/c10/core/DeviceArray.h 2024-08-08T20:48:50.0754098Z inflating: build/lib/torch/include/c10/core/DeviceGuard.h 2024-08-08T20:48:50.0755358Z inflating: build/lib/torch/include/c10/core/DeviceType.h 2024-08-08T20:48:50.0756223Z inflating: build/lib/torch/include/c10/core/DispatchKey.h 2024-08-08T20:48:50.0757587Z inflating: build/lib/torch/include/c10/core/DispatchKeySet.h 2024-08-08T20:48:50.0759093Z inflating: build/lib/torch/include/c10/core/DynamicCast.h 2024-08-08T20:48:50.0760291Z inflating: build/lib/torch/include/c10/core/Event.h 2024-08-08T20:48:50.0761578Z inflating: build/lib/torch/include/c10/core/GeneratorImpl.h 2024-08-08T20:48:50.0762943Z inflating: build/lib/torch/include/c10/core/GradMode.h 2024-08-08T20:48:50.0764179Z inflating: build/lib/torch/include/c10/core/InferenceMode.h 2024-08-08T20:48:50.0764988Z inflating: build/lib/torch/include/c10/core/Layout.h 2024-08-08T20:48:50.0766158Z inflating: build/lib/torch/include/c10/core/MemoryFormat.h 2024-08-08T20:48:50.0766981Z inflating: build/lib/torch/include/c10/core/OptionalRef.h 2024-08-08T20:48:50.0767842Z inflating: build/lib/torch/include/c10/core/PyHandleCache.h 2024-08-08T20:48:50.0769105Z inflating: build/lib/torch/include/c10/core/QEngine.h 2024-08-08T20:48:50.0770050Z inflating: build/lib/torch/include/c10/core/QScheme.h 2024-08-08T20:48:50.0771274Z inflating: build/lib/torch/include/c10/core/RefcountedDeleter.h 2024-08-08T20:48:50.0772295Z inflating: build/lib/torch/include/c10/core/SafePyObject.h 2024-08-08T20:48:50.0773339Z inflating: build/lib/torch/include/c10/core/Scalar.h 2024-08-08T20:48:50.0774517Z inflating: build/lib/torch/include/c10/core/ScalarType.h 2024-08-08T20:48:50.0775749Z inflating: build/lib/torch/include/c10/core/ScalarTypeToTypeMeta.h 2024-08-08T20:48:50.0777002Z inflating: build/lib/torch/include/c10/core/Storage.h 2024-08-08T20:48:50.0778159Z inflating: build/lib/torch/include/c10/core/StorageImpl.h 2024-08-08T20:48:50.0778945Z inflating: build/lib/torch/include/c10/core/Stream.h 2024-08-08T20:48:50.0780045Z inflating: build/lib/torch/include/c10/core/StreamGuard.h 2024-08-08T20:48:50.0781159Z inflating: build/lib/torch/include/c10/core/SymBool.h 2024-08-08T20:48:50.0782152Z inflating: build/lib/torch/include/c10/core/SymFloat.h 2024-08-08T20:48:50.0783086Z inflating: build/lib/torch/include/c10/core/SymInt.h 2024-08-08T20:48:50.0784279Z inflating: build/lib/torch/include/c10/core/SymIntArrayRef.h 2024-08-08T20:48:50.0785133Z inflating: build/lib/torch/include/c10/core/SymNodeImpl.h 2024-08-08T20:48:50.0785999Z inflating: build/lib/torch/include/c10/core/SymbolicShapeMeta.h 2024-08-08T20:48:50.0794939Z inflating: build/lib/torch/include/c10/core/TensorImpl.h 2024-08-08T20:48:50.0797964Z inflating: build/lib/torch/include/c10/core/TensorOptions.h 2024-08-08T20:48:50.0799345Z inflating: build/lib/torch/include/c10/core/UndefinedTensorImpl.h 2024-08-08T20:48:50.0800750Z inflating: build/lib/torch/include/c10/core/WrapDimMinimal.h 2024-08-08T20:48:50.0801988Z inflating: build/lib/torch/include/c10/core/alignment.h 2024-08-08T20:48:50.0803203Z inflating: build/lib/torch/include/c10/core/thread_pool.h 2024-08-08T20:48:50.0804382Z creating: build/lib/torch/include/c10/core/impl/ 2024-08-08T20:48:50.0805114Z inflating: build/lib/torch/include/c10/core/impl/COW.h 2024-08-08T20:48:50.0806128Z inflating: build/lib/torch/include/c10/core/impl/COWDeleter.h 2024-08-08T20:48:50.0807897Z inflating: build/lib/torch/include/c10/core/impl/DeviceGuardImplInterface.h 2024-08-08T20:48:50.0809449Z inflating: build/lib/torch/include/c10/core/impl/FakeGuardImpl.h 2024-08-08T20:48:50.0810446Z inflating: build/lib/torch/include/c10/core/impl/GPUTrace.h 2024-08-08T20:48:50.0811741Z inflating: build/lib/torch/include/c10/core/impl/HermeticPyObjectTLS.h 2024-08-08T20:48:50.0813121Z inflating: build/lib/torch/include/c10/core/impl/InlineDeviceGuard.h 2024-08-08T20:48:50.0814153Z inflating: build/lib/torch/include/c10/core/impl/InlineEvent.h 2024-08-08T20:48:50.0815534Z inflating: build/lib/torch/include/c10/core/impl/InlineStreamGuard.h 2024-08-08T20:48:50.0817038Z inflating: build/lib/torch/include/c10/core/impl/LocalDispatchKeySet.h 2024-08-08T20:48:50.0818659Z inflating: build/lib/torch/include/c10/core/impl/PyInterpreter.h 2024-08-08T20:48:50.0820342Z inflating: build/lib/torch/include/c10/core/impl/PyObjectSlot.h 2024-08-08T20:48:50.0821668Z inflating: build/lib/torch/include/c10/core/impl/PythonDispatcherTLS.h 2024-08-08T20:48:50.0823374Z inflating: build/lib/torch/include/c10/core/impl/SizesAndStrides.h 2024-08-08T20:48:50.0825157Z inflating: build/lib/torch/include/c10/core/impl/TorchDispatchModeTLS.h 2024-08-08T20:48:50.0826737Z inflating: build/lib/torch/include/c10/core/impl/VirtualGuardImpl.h 2024-08-08T20:48:50.0828115Z inflating: build/lib/torch/include/c10/core/impl/alloc_cpu.h 2024-08-08T20:48:50.0829092Z creating: build/lib/torch/include/c10/util/ 2024-08-08T20:48:50.0830201Z inflating: build/lib/torch/include/c10/util/AbortHandler.h 2024-08-08T20:48:50.0831054Z inflating: build/lib/torch/include/c10/util/AlignOf.h 2024-08-08T20:48:50.0832200Z inflating: build/lib/torch/include/c10/util/ApproximateClock.h 2024-08-08T20:48:50.0833165Z inflating: build/lib/torch/include/c10/util/Array.h 2024-08-08T20:48:50.0834120Z inflating: build/lib/torch/include/c10/util/ArrayRef.h 2024-08-08T20:48:50.0835544Z inflating: build/lib/torch/include/c10/util/BFloat16-inl.h 2024-08-08T20:48:50.0837047Z inflating: build/lib/torch/include/c10/util/BFloat16-math.h 2024-08-08T20:48:50.0837851Z inflating: build/lib/torch/include/c10/util/BFloat16.h 2024-08-08T20:48:50.0838831Z inflating: build/lib/torch/include/c10/util/Backtrace.h 2024-08-08T20:48:50.0840056Z inflating: build/lib/torch/include/c10/util/Bitset.h 2024-08-08T20:48:50.0841128Z inflating: build/lib/torch/include/c10/util/C++17.h 2024-08-08T20:48:50.0842266Z inflating: build/lib/torch/include/c10/util/CallOnce.h 2024-08-08T20:48:50.0843782Z inflating: build/lib/torch/include/c10/util/ConstexprCrc.h 2024-08-08T20:48:50.0845095Z inflating: build/lib/torch/include/c10/util/DeadlockDetection.h 2024-08-08T20:48:50.0846562Z inflating: build/lib/torch/include/c10/util/Deprecated.h 2024-08-08T20:48:50.0847800Z inflating: build/lib/torch/include/c10/util/DimVector.h 2024-08-08T20:48:50.0848632Z inflating: build/lib/torch/include/c10/util/DynamicCounter.h 2024-08-08T20:48:50.0851225Z inflating: build/lib/torch/include/c10/util/Exception.h 2024-08-08T20:48:50.0852426Z inflating: build/lib/torch/include/c10/util/ExclusivelyOwned.h 2024-08-08T20:48:50.0854109Z inflating: build/lib/torch/include/c10/util/ExclusivelyOwnedTensorTraits.h 2024-08-08T20:48:50.0855168Z inflating: build/lib/torch/include/c10/util/FbcodeMaps.h 2024-08-08T20:48:50.0856205Z inflating: build/lib/torch/include/c10/util/Flags.h 2024-08-08T20:48:50.0857496Z inflating: build/lib/torch/include/c10/util/Float8_e4m3fn-inl.h 2024-08-08T20:48:50.0859176Z inflating: build/lib/torch/include/c10/util/Float8_e4m3fn.h 2024-08-08T20:48:50.0860932Z inflating: build/lib/torch/include/c10/util/Float8_e4m3fnuz-inl.h 2024-08-08T20:48:50.0862210Z inflating: build/lib/torch/include/c10/util/Float8_e4m3fnuz.h 2024-08-08T20:48:50.0863565Z inflating: build/lib/torch/include/c10/util/Float8_e5m2-inl.h 2024-08-08T20:48:50.0864990Z inflating: build/lib/torch/include/c10/util/Float8_e5m2.h 2024-08-08T20:48:50.0866435Z inflating: build/lib/torch/include/c10/util/Float8_e5m2fnuz-inl.h 2024-08-08T20:48:50.0867933Z inflating: build/lib/torch/include/c10/util/Float8_e5m2fnuz.h 2024-08-08T20:48:50.0869238Z inflating: build/lib/torch/include/c10/util/Float8_fnuz_cvt.h 2024-08-08T20:48:50.0870349Z inflating: build/lib/torch/include/c10/util/FunctionRef.h 2024-08-08T20:48:50.0871807Z inflating: build/lib/torch/include/c10/util/Half-inl.h 2024-08-08T20:48:50.0874498Z inflating: build/lib/torch/include/c10/util/Half.h 2024-08-08T20:48:50.0875633Z inflating: build/lib/torch/include/c10/util/IdWrapper.h 2024-08-08T20:48:50.0876734Z inflating: build/lib/torch/include/c10/util/Lazy.h 2024-08-08T20:48:50.0878026Z inflating: build/lib/torch/include/c10/util/LeftRight.h 2024-08-08T20:48:50.0879169Z inflating: build/lib/torch/include/c10/util/Load.h 2024-08-08T20:48:50.0880892Z inflating: build/lib/torch/include/c10/util/Logging.h 2024-08-08T20:48:50.0882068Z inflating: build/lib/torch/include/c10/util/MathConstants.h 2024-08-08T20:48:50.0883381Z inflating: build/lib/torch/include/c10/util/MaybeOwned.h 2024-08-08T20:48:50.0885127Z inflating: build/lib/torch/include/c10/util/Metaprogramming.h 2024-08-08T20:48:50.0886383Z inflating: build/lib/torch/include/c10/util/Optional.h 2024-08-08T20:48:50.0887756Z inflating: build/lib/torch/include/c10/util/OptionalArrayRef.h 2024-08-08T20:48:50.0888886Z inflating: build/lib/torch/include/c10/util/ParallelGuard.h 2024-08-08T20:48:50.0890297Z inflating: build/lib/torch/include/c10/util/Registry.h 2024-08-08T20:48:50.0891575Z inflating: build/lib/torch/include/c10/util/ScopeExit.h 2024-08-08T20:48:50.0892652Z inflating: build/lib/torch/include/c10/util/SmallBuffer.h 2024-08-08T20:48:50.0897295Z inflating: build/lib/torch/include/c10/util/SmallVector.h 2024-08-08T20:48:50.0898619Z inflating: build/lib/torch/include/c10/util/StringUtil.h 2024-08-08T20:48:50.0899810Z inflating: build/lib/torch/include/c10/util/Synchronized.h 2024-08-08T20:48:50.0901086Z inflating: build/lib/torch/include/c10/util/ThreadLocal.h 2024-08-08T20:48:50.0902541Z inflating: build/lib/torch/include/c10/util/ThreadLocalDebugInfo.h 2024-08-08T20:48:50.0903712Z inflating: build/lib/torch/include/c10/util/Type.h 2024-08-08T20:48:50.0904506Z inflating: build/lib/torch/include/c10/util/TypeCast.h 2024-08-08T20:48:50.0905697Z inflating: build/lib/torch/include/c10/util/TypeIndex.h 2024-08-08T20:48:50.0907680Z inflating: build/lib/torch/include/c10/util/TypeList.h 2024-08-08T20:48:50.0908866Z inflating: build/lib/torch/include/c10/util/TypeSafeSignMath.h 2024-08-08T20:48:50.0910079Z inflating: build/lib/torch/include/c10/util/TypeTraits.h 2024-08-08T20:48:50.0911150Z inflating: build/lib/torch/include/c10/util/Unicode.h 2024-08-08T20:48:50.0912444Z inflating: build/lib/torch/include/c10/util/UniqueVoidPtr.h 2024-08-08T20:48:50.0913707Z inflating: build/lib/torch/include/c10/util/Unroll.h 2024-08-08T20:48:50.0914859Z inflating: build/lib/torch/include/c10/util/WaitCounter.h 2024-08-08T20:48:50.0916031Z inflating: build/lib/torch/include/c10/util/accumulate.h 2024-08-08T20:48:50.0917148Z inflating: build/lib/torch/include/c10/util/bit_cast.h 2024-08-08T20:48:50.0917857Z inflating: build/lib/torch/include/c10/util/bits.h 2024-08-08T20:48:50.0919389Z inflating: build/lib/torch/include/c10/util/complex.h 2024-08-08T20:48:50.0921104Z inflating: build/lib/torch/include/c10/util/complex_math.h 2024-08-08T20:48:50.0922543Z inflating: build/lib/torch/include/c10/util/complex_utils.h 2024-08-08T20:48:50.0923848Z inflating: build/lib/torch/include/c10/util/copysign.h 2024-08-08T20:48:50.0924588Z inflating: build/lib/torch/include/c10/util/env.h 2024-08-08T20:48:50.0929624Z inflating: build/lib/torch/include/c10/util/flat_hash_map.h 2024-08-08T20:48:50.0930995Z inflating: build/lib/torch/include/c10/util/floating_point_utils.h 2024-08-08T20:48:50.0932186Z inflating: build/lib/torch/include/c10/util/generic_math.h 2024-08-08T20:48:50.0933372Z inflating: build/lib/torch/include/c10/util/hash.h 2024-08-08T20:48:50.0935449Z inflating: build/lib/torch/include/c10/util/int128.h 2024-08-08T20:48:50.0939432Z inflating: build/lib/torch/include/c10/util/intrusive_ptr.h 2024-08-08T20:48:50.0940598Z inflating: build/lib/torch/include/c10/util/irange.h 2024-08-08T20:48:50.0943844Z inflating: build/lib/torch/include/c10/util/llvmMathExtras.h 2024-08-08T20:48:50.0945150Z inflating: build/lib/torch/include/c10/util/logging_is_google_glog.h 2024-08-08T20:48:50.0946635Z inflating: build/lib/torch/include/c10/util/logging_is_not_google_glog.h 2024-08-08T20:48:50.0947736Z inflating: build/lib/torch/include/c10/util/numa.h 2024-08-08T20:48:50.0953425Z inflating: build/lib/torch/include/c10/util/order_preserving_flat_hash_map.h 2024-08-08T20:48:50.0955090Z inflating: build/lib/torch/include/c10/util/overloaded.h 2024-08-08T20:48:50.0956464Z inflating: build/lib/torch/include/c10/util/python_stub.h 2024-08-08T20:48:50.0957739Z inflating: build/lib/torch/include/c10/util/qint32.h 2024-08-08T20:48:50.0959462Z inflating: build/lib/torch/include/c10/util/qint8.h 2024-08-08T20:48:50.0960510Z inflating: build/lib/torch/include/c10/util/quint2x4.h 2024-08-08T20:48:50.0961584Z inflating: build/lib/torch/include/c10/util/quint4x2.h 2024-08-08T20:48:50.0962582Z inflating: build/lib/torch/include/c10/util/quint8.h 2024-08-08T20:48:50.0963349Z inflating: build/lib/torch/include/c10/util/safe_numerics.h 2024-08-08T20:48:50.0964200Z inflating: build/lib/torch/include/c10/util/signal_handler.h 2024-08-08T20:48:50.0965416Z inflating: build/lib/torch/include/c10/util/sparse_bitset.h 2024-08-08T20:48:50.0966693Z inflating: build/lib/torch/include/c10/util/ssize.h 2024-08-08T20:48:50.0967875Z inflating: build/lib/torch/include/c10/util/static_tracepoint.h 2024-08-08T20:48:50.0969558Z inflating: build/lib/torch/include/c10/util/static_tracepoint_elfx86.h 2024-08-08T20:48:50.0970988Z inflating: build/lib/torch/include/c10/util/strides.h 2024-08-08T20:48:50.0971776Z inflating: build/lib/torch/include/c10/util/string_utils.h 2024-08-08T20:48:50.0973089Z inflating: build/lib/torch/include/c10/util/string_view.h 2024-08-08T20:48:50.0976573Z inflating: build/lib/torch/include/c10/util/strong_type.h 2024-08-08T20:48:50.0977928Z inflating: build/lib/torch/include/c10/util/tempfile.h 2024-08-08T20:48:50.0978789Z inflating: build/lib/torch/include/c10/util/thread_name.h 2024-08-08T20:48:50.0981307Z inflating: build/lib/torch/include/c10/util/typeid.h 2024-08-08T20:48:50.0982700Z inflating: build/lib/torch/include/c10/util/win32-headers.h 2024-08-08T20:48:50.0983936Z creating: build/lib/torch/include/c10/cuda/ 2024-08-08T20:48:50.0984713Z inflating: build/lib/torch/include/c10/cuda/CUDAAllocatorConfig.h 2024-08-08T20:48:50.0986377Z inflating: build/lib/torch/include/c10/cuda/CUDACachingAllocator.h 2024-08-08T20:48:50.0987909Z inflating: build/lib/torch/include/c10/cuda/CUDADeviceAssertionHost.h 2024-08-08T20:48:50.0989337Z inflating: build/lib/torch/include/c10/cuda/CUDAException.h 2024-08-08T20:48:50.0990876Z inflating: build/lib/torch/include/c10/cuda/CUDAFunctions.h 2024-08-08T20:48:50.0992537Z inflating: build/lib/torch/include/c10/cuda/CUDAGuard.h 2024-08-08T20:48:50.0993823Z inflating: build/lib/torch/include/c10/cuda/CUDAMacros.h 2024-08-08T20:48:50.0995288Z inflating: build/lib/torch/include/c10/cuda/CUDAMathCompat.h 2024-08-08T20:48:50.0996372Z inflating: build/lib/torch/include/c10/cuda/CUDAMiscFunctions.h 2024-08-08T20:48:50.0997732Z inflating: build/lib/torch/include/c10/cuda/CUDAStream.h 2024-08-08T20:48:50.0998941Z inflating: build/lib/torch/include/c10/cuda/CUDAAlgorithm.h 2024-08-08T20:48:50.1000343Z inflating: build/lib/torch/include/c10/cuda/CUDADeviceAssertion.h 2024-08-08T20:48:50.1002084Z inflating: build/lib/torch/include/c10/cuda/CUDAGraphsC10Utils.h 2024-08-08T20:48:50.1003076Z inflating: build/lib/torch/include/c10/cuda/driver_api.h 2024-08-08T20:48:50.1004359Z creating: build/lib/torch/include/c10/cuda/impl/ 2024-08-08T20:48:50.1005661Z inflating: build/lib/torch/include/c10/cuda/impl/CUDAGuardImpl.h 2024-08-08T20:48:50.1007063Z inflating: build/lib/torch/include/c10/cuda/impl/CUDATest.h 2024-08-08T20:48:50.1008173Z inflating: build/lib/torch/include/c10/cuda/impl/cuda_cmake_macros.h 2024-08-08T20:48:50.1009399Z creating: build/lib/torch/include/c10/xpu/ 2024-08-08T20:48:50.1010703Z inflating: build/lib/torch/include/c10/xpu/XPUCachingAllocator.h 2024-08-08T20:48:50.1011792Z inflating: build/lib/torch/include/c10/xpu/XPUDeviceProp.h 2024-08-08T20:48:50.1013079Z inflating: build/lib/torch/include/c10/xpu/XPUException.h 2024-08-08T20:48:50.1014213Z inflating: build/lib/torch/include/c10/xpu/XPUFunctions.h 2024-08-08T20:48:50.1015221Z inflating: build/lib/torch/include/c10/xpu/XPUMacros.h 2024-08-08T20:48:50.1015981Z inflating: build/lib/torch/include/c10/xpu/XPUStream.h 2024-08-08T20:48:50.1016833Z creating: build/lib/torch/include/c10/xpu/impl/ 2024-08-08T20:48:50.1017926Z inflating: build/lib/torch/include/c10/xpu/impl/XPUGuardImpl.h 2024-08-08T20:48:50.1019058Z creating: build/lib/torch/include/torch/ 2024-08-08T20:48:50.1019740Z inflating: build/lib/torch/include/torch/custom_class.h 2024-08-08T20:48:50.1020559Z inflating: build/lib/torch/include/torch/custom_class_detail.h 2024-08-08T20:48:50.1023622Z inflating: build/lib/torch/include/torch/library.h 2024-08-08T20:48:50.1024763Z inflating: build/lib/torch/include/torch/script.h 2024-08-08T20:48:50.1026002Z inflating: build/lib/torch/include/torch/extension.h 2024-08-08T20:48:50.1027098Z creating: build/lib/torch/include/torch/csrc/ 2024-08-08T20:48:50.1028195Z inflating: build/lib/torch/include/torch/csrc/Export.h 2024-08-08T20:48:50.1029592Z inflating: build/lib/torch/include/torch/csrc/CudaIPCTypes.h 2024-08-08T20:48:50.1030940Z inflating: build/lib/torch/include/torch/csrc/DataLoader.h 2024-08-08T20:48:50.1032077Z inflating: build/lib/torch/include/torch/csrc/Device.h 2024-08-08T20:48:50.1032822Z inflating: build/lib/torch/include/torch/csrc/Dtype.h 2024-08-08T20:48:50.1033971Z inflating: build/lib/torch/include/torch/csrc/DynamicTypes.h 2024-08-08T20:48:50.1035286Z inflating: build/lib/torch/include/torch/csrc/Event.h 2024-08-08T20:48:50.1036534Z inflating: build/lib/torch/include/torch/csrc/Exceptions.h 2024-08-08T20:48:50.1037855Z inflating: build/lib/torch/include/torch/csrc/Generator.h 2024-08-08T20:48:50.1039048Z inflating: build/lib/torch/include/torch/csrc/Layout.h 2024-08-08T20:48:50.1040375Z inflating: build/lib/torch/include/torch/csrc/MemoryFormat.h 2024-08-08T20:48:50.1041730Z inflating: build/lib/torch/include/torch/csrc/Module.h 2024-08-08T20:48:50.1043109Z inflating: build/lib/torch/include/torch/csrc/PyInterpreter.h 2024-08-08T20:48:50.1044417Z inflating: build/lib/torch/include/torch/csrc/QScheme.h 2024-08-08T20:48:50.1045585Z inflating: build/lib/torch/include/torch/csrc/Size.h 2024-08-08T20:48:50.1046926Z inflating: build/lib/torch/include/torch/csrc/Storage.h 2024-08-08T20:48:50.1048099Z inflating: build/lib/torch/include/torch/csrc/StorageMethods.h 2024-08-08T20:48:50.1049523Z inflating: build/lib/torch/include/torch/csrc/StorageSharing.h 2024-08-08T20:48:50.1050866Z inflating: build/lib/torch/include/torch/csrc/Stream.h 2024-08-08T20:48:50.1051855Z inflating: build/lib/torch/include/torch/csrc/THConcat.h 2024-08-08T20:48:50.1052852Z inflating: build/lib/torch/include/torch/csrc/THP.h 2024-08-08T20:48:50.1053872Z inflating: build/lib/torch/include/torch/csrc/TypeInfo.h 2024-08-08T20:48:50.1055041Z inflating: build/lib/torch/include/torch/csrc/Types.h 2024-08-08T20:48:50.1056473Z inflating: build/lib/torch/include/torch/csrc/copy_utils.h 2024-08-08T20:48:50.1057957Z inflating: build/lib/torch/include/torch/csrc/itt_wrapper.h 2024-08-08T20:48:50.1059556Z inflating: build/lib/torch/include/torch/csrc/python_dimname.h 2024-08-08T20:48:50.1060717Z inflating: build/lib/torch/include/torch/csrc/python_headers.h 2024-08-08T20:48:50.1061974Z inflating: build/lib/torch/include/torch/csrc/serialization.h 2024-08-08T20:48:50.1063170Z inflating: build/lib/torch/include/torch/csrc/utils.h 2024-08-08T20:48:50.1063901Z creating: build/lib/torch/include/torch/csrc/api/ 2024-08-08T20:48:50.1064972Z creating: build/lib/torch/include/torch/csrc/api/include/ 2024-08-08T20:48:50.1066371Z creating: build/lib/torch/include/torch/csrc/api/include/torch/ 2024-08-08T20:48:50.1067533Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/all.h 2024-08-08T20:48:50.1068548Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/arg.h 2024-08-08T20:48:50.1070160Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/autograd.h 2024-08-08T20:48:50.1071660Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/cuda.h 2024-08-08T20:48:50.1073486Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data.h 2024-08-08T20:48:50.1074674Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/enum.h 2024-08-08T20:48:50.1076193Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/expanding_array.h 2024-08-08T20:48:50.1077669Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/fft.h 2024-08-08T20:48:50.1079141Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/imethod.h 2024-08-08T20:48:50.1080419Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/jit.h 2024-08-08T20:48:50.1081471Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/linalg.h 2024-08-08T20:48:50.1082955Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/mps.h 2024-08-08T20:48:50.1084462Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nested.h 2024-08-08T20:48:50.1085914Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn.h 2024-08-08T20:48:50.1087673Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim.h 2024-08-08T20:48:50.1089232Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/ordered_dict.h 2024-08-08T20:48:50.1090925Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/python.h 2024-08-08T20:48:50.1092561Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/serialize.h 2024-08-08T20:48:50.1094057Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/sparse.h 2024-08-08T20:48:50.1095813Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/special.h 2024-08-08T20:48:50.1097409Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/torch.h 2024-08-08T20:48:50.1098641Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/types.h 2024-08-08T20:48:50.1100121Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/utils.h 2024-08-08T20:48:50.1101663Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/xpu.h 2024-08-08T20:48:50.1103064Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/version.h 2024-08-08T20:48:50.1104474Z creating: build/lib/torch/include/torch/csrc/api/include/torch/data/ 2024-08-08T20:48:50.1105575Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/dataloader.h 2024-08-08T20:48:50.1107218Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h 2024-08-08T20:48:50.1109052Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/datasets.h 2024-08-08T20:48:50.1110747Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/example.h 2024-08-08T20:48:50.1112515Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/iterator.h 2024-08-08T20:48:50.1114437Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/samplers.h 2024-08-08T20:48:50.1116140Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/transforms.h 2024-08-08T20:48:50.1118028Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/worker_exception.h 2024-08-08T20:48:50.1119936Z creating: build/lib/torch/include/torch/csrc/api/include/torch/data/dataloader/ 2024-08-08T20:48:50.1121657Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h 2024-08-08T20:48:50.1123646Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h 2024-08-08T20:48:50.1125665Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/dataloader/stateless.h 2024-08-08T20:48:50.1127503Z creating: build/lib/torch/include/torch/csrc/api/include/torch/data/datasets/ 2024-08-08T20:48:50.1129306Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/datasets/base.h 2024-08-08T20:48:50.1131101Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/datasets/chunk.h 2024-08-08T20:48:50.1132883Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/datasets/map.h 2024-08-08T20:48:50.1134543Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/datasets/mnist.h 2024-08-08T20:48:50.1136210Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/datasets/shared.h 2024-08-08T20:48:50.1137988Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/datasets/stateful.h 2024-08-08T20:48:50.1139964Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/datasets/tensor.h 2024-08-08T20:48:50.1141328Z creating: build/lib/torch/include/torch/csrc/api/include/torch/data/detail/ 2024-08-08T20:48:50.1142548Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/detail/data_shuttle.h 2024-08-08T20:48:50.1144582Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/detail/queue.h 2024-08-08T20:48:50.1146676Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/detail/sequencers.h 2024-08-08T20:48:50.1148348Z creating: build/lib/torch/include/torch/csrc/api/include/torch/data/samplers/ 2024-08-08T20:48:50.1149851Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/samplers/base.h 2024-08-08T20:48:50.1151467Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/samplers/custom_batch_request.h 2024-08-08T20:48:50.1153309Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/samplers/distributed.h 2024-08-08T20:48:50.1155186Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/samplers/random.h 2024-08-08T20:48:50.1157115Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/samplers/sequential.h 2024-08-08T20:48:50.1159247Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/samplers/serialize.h 2024-08-08T20:48:50.1161232Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/samplers/stream.h 2024-08-08T20:48:50.1163135Z creating: build/lib/torch/include/torch/csrc/api/include/torch/data/transforms/ 2024-08-08T20:48:50.1164568Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/transforms/base.h 2024-08-08T20:48:50.1165997Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/transforms/collate.h 2024-08-08T20:48:50.1167762Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/transforms/lambda.h 2024-08-08T20:48:50.1169657Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/transforms/stack.h 2024-08-08T20:48:50.1171382Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/data/transforms/tensor.h 2024-08-08T20:48:50.1172544Z creating: build/lib/torch/include/torch/csrc/api/include/torch/detail/ 2024-08-08T20:48:50.1174183Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/detail/TensorDataContainer.h 2024-08-08T20:48:50.1176331Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/detail/static.h 2024-08-08T20:48:50.1177752Z creating: build/lib/torch/include/torch/csrc/api/include/torch/nn/ 2024-08-08T20:48:50.1178817Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/cloneable.h 2024-08-08T20:48:50.1180637Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional.h 2024-08-08T20:48:50.1182233Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/init.h 2024-08-08T20:48:50.1183611Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/module.h 2024-08-08T20:48:50.1185279Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules.h 2024-08-08T20:48:50.1186694Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options.h 2024-08-08T20:48:50.1188785Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/pimpl-inl.h 2024-08-08T20:48:50.1190408Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/pimpl.h 2024-08-08T20:48:50.1191055Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/utils.h 2024-08-08T20:48:50.1191868Z creating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/ 2024-08-08T20:48:50.1192502Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/activation.h 2024-08-08T20:48:50.1193097Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/batchnorm.h 2024-08-08T20:48:50.1193878Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/conv.h 2024-08-08T20:48:50.1194625Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/distance.h 2024-08-08T20:48:50.1195510Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/dropout.h 2024-08-08T20:48:50.1196149Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/embedding.h 2024-08-08T20:48:50.1197006Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/fold.h 2024-08-08T20:48:50.1197930Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/instancenorm.h 2024-08-08T20:48:50.1198644Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/linear.h 2024-08-08T20:48:50.1199200Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/loss.h 2024-08-08T20:48:50.1200055Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/normalization.h 2024-08-08T20:48:50.1200765Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h 2024-08-08T20:48:50.1201665Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/pixelshuffle.h 2024-08-08T20:48:50.1202539Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/pooling.h 2024-08-08T20:48:50.1203418Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/upsampling.h 2024-08-08T20:48:50.1204256Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/functional/vision.h 2024-08-08T20:48:50.1204929Z creating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/ 2024-08-08T20:48:50.1205690Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/activation.h 2024-08-08T20:48:50.1206467Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/adaptive.h 2024-08-08T20:48:50.1207352Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/batchnorm.h 2024-08-08T20:48:50.1208049Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/conv.h 2024-08-08T20:48:50.1208806Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/distance.h 2024-08-08T20:48:50.1209639Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/dropout.h 2024-08-08T20:48:50.1210350Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/embedding.h 2024-08-08T20:48:50.1210878Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/fold.h 2024-08-08T20:48:50.1211543Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/instancenorm.h 2024-08-08T20:48:50.1212465Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/linear.h 2024-08-08T20:48:50.1213194Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/loss.h 2024-08-08T20:48:50.1213981Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/normalization.h 2024-08-08T20:48:50.1214696Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/padding.h 2024-08-08T20:48:50.1215496Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/pixelshuffle.h 2024-08-08T20:48:50.1216262Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/pooling.h 2024-08-08T20:48:50.1216935Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/rnn.h 2024-08-08T20:48:50.1217640Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/transformer.h 2024-08-08T20:48:50.1218650Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/transformercoder.h 2024-08-08T20:48:50.1219519Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/transformerlayer.h 2024-08-08T20:48:50.1220281Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/upsampling.h 2024-08-08T20:48:50.1221103Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/options/vision.h 2024-08-08T20:48:50.1221641Z creating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/ 2024-08-08T20:48:50.1222448Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/_functions.h 2024-08-08T20:48:50.1223370Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/activation.h 2024-08-08T20:48:50.1224074Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/adaptive.h 2024-08-08T20:48:50.1224745Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/batchnorm.h 2024-08-08T20:48:50.1225556Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/common.h 2024-08-08T20:48:50.1226150Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/conv.h 2024-08-08T20:48:50.1226920Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/distance.h 2024-08-08T20:48:50.1227626Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/dropout.h 2024-08-08T20:48:50.1228179Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/embedding.h 2024-08-08T20:48:50.1228688Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/fold.h 2024-08-08T20:48:50.1229377Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/instancenorm.h 2024-08-08T20:48:50.1230301Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/linear.h 2024-08-08T20:48:50.1230893Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/loss.h 2024-08-08T20:48:50.1232018Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/normalization.h 2024-08-08T20:48:50.1233808Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/padding.h 2024-08-08T20:48:50.1235038Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/pixelshuffle.h 2024-08-08T20:48:50.1237782Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/pooling.h 2024-08-08T20:48:50.1239940Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/rnn.h 2024-08-08T20:48:50.1241765Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/transformer.h 2024-08-08T20:48:50.1243174Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/transformercoder.h 2024-08-08T20:48:50.1244750Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/transformerlayer.h 2024-08-08T20:48:50.1245730Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/upsampling.h 2024-08-08T20:48:50.1246849Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/utils.h 2024-08-08T20:48:50.1247562Z creating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/ 2024-08-08T20:48:50.1250064Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/any.h 2024-08-08T20:48:50.1251489Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/any_module_holder.h 2024-08-08T20:48:50.1252844Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/any_value.h 2024-08-08T20:48:50.1254259Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/functional.h 2024-08-08T20:48:50.1256100Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/moduledict.h 2024-08-08T20:48:50.1257952Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/modulelist.h 2024-08-08T20:48:50.1259358Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/named_any.h 2024-08-08T20:48:50.1260957Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/parameterdict.h 2024-08-08T20:48:50.1262372Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/parameterlist.h 2024-08-08T20:48:50.1265125Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/modules/container/sequential.h 2024-08-08T20:48:50.1265632Z creating: build/lib/torch/include/torch/csrc/api/include/torch/nn/parallel/ 2024-08-08T20:48:50.1267540Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/parallel/data_parallel.h 2024-08-08T20:48:50.1268091Z creating: build/lib/torch/include/torch/csrc/api/include/torch/nn/utils/ 2024-08-08T20:48:50.1269354Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/utils/clip_grad.h 2024-08-08T20:48:50.1270616Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/utils/convert_parameters.h 2024-08-08T20:48:50.1272918Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/nn/utils/rnn.h 2024-08-08T20:48:50.1273446Z creating: build/lib/torch/include/torch/csrc/api/include/torch/optim/ 2024-08-08T20:48:50.1274367Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/adagrad.h 2024-08-08T20:48:50.1275688Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/adam.h 2024-08-08T20:48:50.1276855Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/adamw.h 2024-08-08T20:48:50.1278045Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/lbfgs.h 2024-08-08T20:48:50.1279691Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/optimizer.h 2024-08-08T20:48:50.1280815Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/rmsprop.h 2024-08-08T20:48:50.1283002Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/serialize.h 2024-08-08T20:48:50.1283723Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/sgd.h 2024-08-08T20:48:50.1284476Z creating: build/lib/torch/include/torch/csrc/api/include/torch/optim/schedulers/ 2024-08-08T20:48:50.1285618Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/schedulers/lr_scheduler.h 2024-08-08T20:48:50.1287206Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/schedulers/reduce_on_plateau_scheduler.h 2024-08-08T20:48:50.1288512Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/optim/schedulers/step_lr.h 2024-08-08T20:48:50.1289125Z creating: build/lib/torch/include/torch/csrc/api/include/torch/serialize/ 2024-08-08T20:48:50.1290170Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/serialize/archive.h 2024-08-08T20:48:50.1291498Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/serialize/input-archive.h 2024-08-08T20:48:50.1292633Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/serialize/output-archive.h 2024-08-08T20:48:50.1293168Z inflating: build/lib/torch/include/torch/csrc/api/include/torch/serialize/tensor.h 2024-08-08T20:48:50.1293446Z creating: build/lib/torch/include/torch/csrc/autograd/ 2024-08-08T20:48:50.1296291Z inflating: build/lib/torch/include/torch/csrc/autograd/FunctionsManual.h 2024-08-08T20:48:50.1296893Z inflating: build/lib/torch/include/torch/csrc/autograd/InferenceMode.h 2024-08-08T20:48:50.1299234Z inflating: build/lib/torch/include/torch/csrc/autograd/VariableTypeUtils.h 2024-08-08T20:48:50.1299837Z inflating: build/lib/torch/include/torch/csrc/autograd/anomaly_mode.h 2024-08-08T20:48:50.1301547Z inflating: build/lib/torch/include/torch/csrc/autograd/autograd.h 2024-08-08T20:48:50.1302677Z inflating: build/lib/torch/include/torch/csrc/autograd/autograd_not_implemented_fallback.h 2024-08-08T20:48:50.1303205Z inflating: build/lib/torch/include/torch/csrc/autograd/cpp_hook.h 2024-08-08T20:48:50.1306362Z inflating: build/lib/torch/include/torch/csrc/autograd/custom_function.h 2024-08-08T20:48:50.1306965Z inflating: build/lib/torch/include/torch/csrc/autograd/edge.h 2024-08-08T20:48:50.1309055Z inflating: build/lib/torch/include/torch/csrc/autograd/engine.h 2024-08-08T20:48:50.1310878Z inflating: build/lib/torch/include/torch/csrc/autograd/forward_grad.h 2024-08-08T20:48:50.1314428Z inflating: build/lib/torch/include/torch/csrc/autograd/function.h 2024-08-08T20:48:50.1315113Z inflating: build/lib/torch/include/torch/csrc/autograd/function_hook.h 2024-08-08T20:48:50.1315945Z inflating: build/lib/torch/include/torch/csrc/autograd/grad_mode.h 2024-08-08T20:48:50.1318166Z inflating: build/lib/torch/include/torch/csrc/autograd/graph_task.h 2024-08-08T20:48:50.1319039Z inflating: build/lib/torch/include/torch/csrc/autograd/input_buffer.h 2024-08-08T20:48:50.1320280Z inflating: build/lib/torch/include/torch/csrc/autograd/input_metadata.h 2024-08-08T20:48:50.1321407Z inflating: build/lib/torch/include/torch/csrc/autograd/jit_decomp_interface.h 2024-08-08T20:48:50.1322057Z inflating: build/lib/torch/include/torch/csrc/autograd/profiler.h 2024-08-08T20:48:50.1324069Z inflating: build/lib/torch/include/torch/csrc/autograd/profiler_kineto.h 2024-08-08T20:48:50.1325927Z inflating: build/lib/torch/include/torch/csrc/autograd/profiler_legacy.h 2024-08-08T20:48:50.1326890Z inflating: build/lib/torch/include/torch/csrc/autograd/profiler_python.h 2024-08-08T20:48:50.1327922Z inflating: build/lib/torch/include/torch/csrc/autograd/python_anomaly_mode.h 2024-08-08T20:48:50.1328546Z inflating: build/lib/torch/include/torch/csrc/autograd/python_autograd.h 2024-08-08T20:48:50.1329976Z inflating: build/lib/torch/include/torch/csrc/autograd/python_cpp_function.h 2024-08-08T20:48:50.1330886Z inflating: build/lib/torch/include/torch/csrc/autograd/python_engine.h 2024-08-08T20:48:50.1331623Z inflating: build/lib/torch/include/torch/csrc/autograd/python_enum_tag.h 2024-08-08T20:48:50.1332518Z extracting: build/lib/torch/include/torch/csrc/autograd/python_fft_functions.h 2024-08-08T20:48:50.1333911Z inflating: build/lib/torch/include/torch/csrc/autograd/python_function.h 2024-08-08T20:48:50.1334900Z inflating: build/lib/torch/include/torch/csrc/autograd/python_hook.h 2024-08-08T20:48:50.1335998Z inflating: build/lib/torch/include/torch/csrc/autograd/python_legacy_variable.h 2024-08-08T20:48:50.1337098Z inflating: build/lib/torch/include/torch/csrc/autograd/python_linalg_functions.h 2024-08-08T20:48:50.1338348Z inflating: build/lib/torch/include/torch/csrc/autograd/python_nested_functions.h 2024-08-08T20:48:50.1339067Z inflating: build/lib/torch/include/torch/csrc/autograd/python_nn_functions.h 2024-08-08T20:48:50.1339691Z inflating: build/lib/torch/include/torch/csrc/autograd/python_saved_variable_hooks.h 2024-08-08T20:48:50.1341157Z inflating: build/lib/torch/include/torch/csrc/autograd/python_sparse_functions.h 2024-08-08T20:48:50.1342008Z inflating: build/lib/torch/include/torch/csrc/autograd/python_special_functions.h 2024-08-08T20:48:50.1342680Z inflating: build/lib/torch/include/torch/csrc/autograd/python_torch_functions.h 2024-08-08T20:48:50.1343632Z inflating: build/lib/torch/include/torch/csrc/autograd/python_variable.h 2024-08-08T20:48:50.1344818Z inflating: build/lib/torch/include/torch/csrc/autograd/python_variable_indexing.h 2024-08-08T20:48:50.1345656Z inflating: build/lib/torch/include/torch/csrc/autograd/record_function_ops.h 2024-08-08T20:48:50.1347548Z inflating: build/lib/torch/include/torch/csrc/autograd/saved_variable.h 2024-08-08T20:48:50.1348190Z inflating: build/lib/torch/include/torch/csrc/autograd/saved_variable_hooks.h 2024-08-08T20:48:50.1348977Z inflating: build/lib/torch/include/torch/csrc/autograd/symbolic.h 2024-08-08T20:48:50.1353580Z inflating: build/lib/torch/include/torch/csrc/autograd/variable.h 2024-08-08T20:48:50.1354151Z inflating: build/lib/torch/include/torch/csrc/autograd/variable_info.h 2024-08-08T20:48:50.1354702Z creating: build/lib/torch/include/torch/csrc/autograd/functions/ 2024-08-08T20:48:50.1357970Z inflating: build/lib/torch/include/torch/csrc/autograd/functions/accumulate_grad.h 2024-08-08T20:48:50.1359114Z inflating: build/lib/torch/include/torch/csrc/autograd/functions/basic_ops.h 2024-08-08T20:48:50.1360066Z inflating: build/lib/torch/include/torch/csrc/autograd/functions/comm.h 2024-08-08T20:48:50.1360734Z inflating: build/lib/torch/include/torch/csrc/autograd/functions/pybind.h 2024-08-08T20:48:50.1362693Z inflating: build/lib/torch/include/torch/csrc/autograd/functions/tensor.h 2024-08-08T20:48:50.1363795Z inflating: build/lib/torch/include/torch/csrc/autograd/functions/utils.h 2024-08-08T20:48:50.1364418Z creating: build/lib/torch/include/torch/csrc/autograd/generated/ 2024-08-08T20:48:50.1365442Z inflating: build/lib/torch/include/torch/csrc/autograd/generated/python_functions.h 2024-08-08T20:48:50.1366439Z inflating: build/lib/torch/include/torch/csrc/autograd/generated/python_return_types.h 2024-08-08T20:48:50.1367329Z inflating: build/lib/torch/include/torch/csrc/autograd/generated/VariableType.h 2024-08-08T20:48:50.1393075Z inflating: build/lib/torch/include/torch/csrc/autograd/generated/Functions.h 2024-08-08T20:48:50.1397038Z inflating: build/lib/torch/include/torch/csrc/autograd/generated/variable_factories.h 2024-08-08T20:48:50.1400017Z inflating: build/lib/torch/include/torch/csrc/autograd/generated/ViewFuncs.h 2024-08-08T20:48:50.1400703Z creating: build/lib/torch/include/torch/csrc/autograd/utils/ 2024-08-08T20:48:50.1401328Z inflating: build/lib/torch/include/torch/csrc/autograd/utils/error_messages.h 2024-08-08T20:48:50.1402299Z inflating: build/lib/torch/include/torch/csrc/autograd/utils/grad_layout_contract.h 2024-08-08T20:48:50.1403216Z inflating: build/lib/torch/include/torch/csrc/autograd/utils/lambda_post_hook.h 2024-08-08T20:48:50.1404308Z inflating: build/lib/torch/include/torch/csrc/autograd/utils/python_arg_parsing.h 2024-08-08T20:48:50.1405143Z inflating: build/lib/torch/include/torch/csrc/autograd/utils/warnings.h 2024-08-08T20:48:50.1406459Z inflating: build/lib/torch/include/torch/csrc/autograd/utils/wrap_outputs.h 2024-08-08T20:48:50.1406807Z creating: build/lib/torch/include/torch/csrc/cuda/ 2024-08-08T20:48:50.1408418Z inflating: build/lib/torch/include/torch/csrc/cuda/CUDAPluggableAllocator.h 2024-08-08T20:48:50.1409339Z inflating: build/lib/torch/include/torch/csrc/cuda/Event.h 2024-08-08T20:48:50.1409893Z inflating: build/lib/torch/include/torch/csrc/cuda/Module.h 2024-08-08T20:48:50.1410299Z inflating: build/lib/torch/include/torch/csrc/cuda/Stream.h 2024-08-08T20:48:50.1411057Z inflating: build/lib/torch/include/torch/csrc/cuda/THCP.h 2024-08-08T20:48:50.1412091Z inflating: build/lib/torch/include/torch/csrc/cuda/comm.h 2024-08-08T20:48:50.1413166Z inflating: build/lib/torch/include/torch/csrc/cuda/device_set.h 2024-08-08T20:48:50.1414247Z inflating: build/lib/torch/include/torch/csrc/cuda/memory_snapshot.h 2024-08-08T20:48:50.1415356Z inflating: build/lib/torch/include/torch/csrc/cuda/nccl.h 2024-08-08T20:48:50.1416726Z inflating: build/lib/torch/include/torch/csrc/cuda/python_comm.h 2024-08-08T20:48:50.1417213Z inflating: build/lib/torch/include/torch/csrc/cuda/python_nccl.h 2024-08-08T20:48:50.1417741Z creating: build/lib/torch/include/torch/csrc/distributed/ 2024-08-08T20:48:50.1418337Z creating: build/lib/torch/include/torch/csrc/distributed/c10d/ 2024-08-08T20:48:50.1420848Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/TraceUtils.h 2024-08-08T20:48:50.1421442Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/c10d.h 2024-08-08T20:48:50.1422304Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/debug.h 2024-08-08T20:48:50.1423478Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/error.h 2024-08-08T20:48:50.1424115Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/exception.h 2024-08-08T20:48:50.1425046Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/logging.h 2024-08-08T20:48:50.1426035Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/python_comm_hook.h 2024-08-08T20:48:50.1427109Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/socket.h 2024-08-08T20:48:50.1429286Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/Backend.hpp 2024-08-08T20:48:50.1429882Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/Backoff.hpp 2024-08-08T20:48:50.1431719Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/CUDASymmetricMemory.hpp 2024-08-08T20:48:50.1432335Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/DMAConnectivity.hpp 2024-08-08T20:48:50.1434089Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/FakeProcessGroup.hpp 2024-08-08T20:48:50.1434655Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/FileStore.hpp 2024-08-08T20:48:50.1435679Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/Functional.hpp 2024-08-08T20:48:50.1437011Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/GlooDeviceFactory.hpp 2024-08-08T20:48:50.1438038Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/GroupRegistry.hpp 2024-08-08T20:48:50.1438917Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/HashStore.hpp 2024-08-08T20:48:50.1442933Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/NCCLUtils.hpp 2024-08-08T20:48:50.1444436Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/ParamCommsUtils.hpp 2024-08-08T20:48:50.1445339Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/PrefixStore.hpp 2024-08-08T20:48:50.1448444Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/ProcessGroup.hpp 2024-08-08T20:48:50.1450691Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/ProcessGroupGloo.hpp 2024-08-08T20:48:50.1452178Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/ProcessGroupMPI.hpp 2024-08-08T20:48:50.1457051Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/ProcessGroupNCCL.hpp 2024-08-08T20:48:50.1458165Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/ProcessGroupRoundRobin.hpp 2024-08-08T20:48:50.1460544Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/ProcessGroupUCC.hpp 2024-08-08T20:48:50.1461852Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/ProcessGroupWrapper.hpp 2024-08-08T20:48:50.1463216Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/PyProcessGroup.hpp 2024-08-08T20:48:50.1464259Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/RankLocal.hpp 2024-08-08T20:48:50.1465669Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/Store.hpp 2024-08-08T20:48:50.1467291Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/SymmetricMemory.hpp 2024-08-08T20:48:50.1468603Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/TCPStore.hpp 2024-08-08T20:48:50.1469632Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/TCPStoreBackend.hpp 2024-08-08T20:48:50.1470948Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/Types.hpp 2024-08-08T20:48:50.1472081Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/UCCTracing.hpp 2024-08-08T20:48:50.1473627Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/UCCUtils.hpp 2024-08-08T20:48:50.1474713Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/UnixSockUtils.hpp 2024-08-08T20:48:50.1477546Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/Utils.hpp 2024-08-08T20:48:50.1478238Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/WinSockUtils.hpp 2024-08-08T20:48:50.1479541Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/Work.hpp 2024-08-08T20:48:50.1480910Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/comm.hpp 2024-08-08T20:48:50.1482049Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/default_comm_hooks.hpp 2024-08-08T20:48:50.1483486Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/intra_node_comm.hpp 2024-08-08T20:48:50.1484882Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/logger.hpp 2024-08-08T20:48:50.1488324Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/reducer.hpp 2024-08-08T20:48:50.1489431Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/reducer_timer.hpp 2024-08-08T20:48:50.1490461Z inflating: build/lib/torch/include/torch/csrc/distributed/c10d/sequence_num.hpp 2024-08-08T20:48:50.1490980Z creating: build/lib/torch/include/torch/csrc/distributed/rpc/ 2024-08-08T20:48:50.1492066Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/agent_utils.h 2024-08-08T20:48:50.1493790Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/message.h 2024-08-08T20:48:50.1494997Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/py_rref.h 2024-08-08T20:48:50.1495866Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/python_call.h 2024-08-08T20:48:50.1497049Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/python_functions.h 2024-08-08T20:48:50.1498195Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/python_remote_call.h 2024-08-08T20:48:50.1498890Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/python_resp.h 2024-08-08T20:48:50.1500242Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/python_rpc_handler.h 2024-08-08T20:48:50.1501542Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/request_callback.h 2024-08-08T20:48:50.1502581Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/request_callback_impl.h 2024-08-08T20:48:50.1504096Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/request_callback_no_python.h 2024-08-08T20:48:50.1504672Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/rpc.h 2024-08-08T20:48:50.1507114Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/rpc_agent.h 2024-08-08T20:48:50.1508065Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/rpc_command_base.h 2024-08-08T20:48:50.1510660Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/rref_context.h 2024-08-08T20:48:50.1513111Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/rref_impl.h 2024-08-08T20:48:50.1514231Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/rref_proto.h 2024-08-08T20:48:50.1515359Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/script_call.h 2024-08-08T20:48:50.1516508Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/script_remote_call.h 2024-08-08T20:48:50.1517374Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/script_resp.h 2024-08-08T20:48:50.1520082Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/tensorpipe_agent.h 2024-08-08T20:48:50.1521388Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/tensorpipe_utils.h 2024-08-08T20:48:50.1522422Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/torchscript_functions.h 2024-08-08T20:48:50.1523233Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/types.h 2024-08-08T20:48:50.1524571Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/unpickled_python_call.h 2024-08-08T20:48:50.1525799Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/unpickled_python_remote_call.h 2024-08-08T20:48:50.1526717Z inflating: build/lib/torch/include/torch/csrc/distributed/rpc/utils.h 2024-08-08T20:48:50.1527289Z creating: build/lib/torch/include/torch/csrc/distributed/autograd/ 2024-08-08T20:48:50.1527877Z creating: build/lib/torch/include/torch/csrc/distributed/autograd/context/ 2024-08-08T20:48:50.1529392Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/context/container.h 2024-08-08T20:48:50.1531014Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/context/context.h 2024-08-08T20:48:50.1531583Z creating: build/lib/torch/include/torch/csrc/distributed/autograd/functions/ 2024-08-08T20:48:50.1533105Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/functions/recvrpc_backward.h 2024-08-08T20:48:50.1534178Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/functions/sendrpc_backward.h 2024-08-08T20:48:50.1535123Z creating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/ 2024-08-08T20:48:50.1536341Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/autograd_metadata.h 2024-08-08T20:48:50.1537547Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/cleanup_autograd_context_req.h 2024-08-08T20:48:50.1538790Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/cleanup_autograd_context_resp.h 2024-08-08T20:48:50.1539702Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/propagate_gradients_req.h 2024-08-08T20:48:50.1540915Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/propagate_gradients_resp.h 2024-08-08T20:48:50.1542029Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/rpc_with_autograd.h 2024-08-08T20:48:50.1543254Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/rpc_with_profiling_req.h 2024-08-08T20:48:50.1544448Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/rpc_with_profiling_resp.h 2024-08-08T20:48:50.1545408Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/rref_backward_req.h 2024-08-08T20:48:50.1546473Z inflating: build/lib/torch/include/torch/csrc/distributed/autograd/rpc_messages/rref_backward_resp.h 2024-08-08T20:48:50.1546758Z creating: build/lib/torch/include/torch/csrc/dynamo/ 2024-08-08T20:48:50.1547145Z inflating: build/lib/torch/include/torch/csrc/dynamo/cache_entry.h 2024-08-08T20:48:50.1549890Z inflating: build/lib/torch/include/torch/csrc/dynamo/compiled_autograd.h 2024-08-08T20:48:50.1550484Z inflating: build/lib/torch/include/torch/csrc/dynamo/cpp_shim.h 2024-08-08T20:48:50.1551272Z inflating: build/lib/torch/include/torch/csrc/dynamo/cpython_defs.h 2024-08-08T20:48:50.1551826Z inflating: build/lib/torch/include/torch/csrc/dynamo/cpython_includes.h 2024-08-08T20:48:50.1552845Z inflating: build/lib/torch/include/torch/csrc/dynamo/debug_macros.h 2024-08-08T20:48:50.1553469Z extracting: build/lib/torch/include/torch/csrc/dynamo/eval_frame.h 2024-08-08T20:48:50.1555102Z inflating: build/lib/torch/include/torch/csrc/dynamo/extra_state.h 2024-08-08T20:48:50.1555952Z inflating: build/lib/torch/include/torch/csrc/dynamo/framelocals_mapping.h 2024-08-08T20:48:50.1557060Z inflating: build/lib/torch/include/torch/csrc/dynamo/guards.h 2024-08-08T20:48:50.1558121Z inflating: build/lib/torch/include/torch/csrc/dynamo/init.h 2024-08-08T20:48:50.1559536Z inflating: build/lib/torch/include/torch/csrc/dynamo/python_compiled_autograd.h 2024-08-08T20:48:50.1560580Z inflating: build/lib/torch/include/torch/csrc/dynamo/utils.h 2024-08-08T20:48:50.1561066Z creating: build/lib/torch/include/torch/csrc/inductor/ 2024-08-08T20:48:50.1562141Z inflating: build/lib/torch/include/torch/csrc/inductor/inductor_ops.h 2024-08-08T20:48:50.1562956Z creating: build/lib/torch/include/torch/csrc/inductor/aoti_runner/ 2024-08-08T20:48:50.1564100Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runner/model_container_runner.h 2024-08-08T20:48:50.1565048Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runner/model_container_runner_cpu.h 2024-08-08T20:48:50.1566324Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runner/model_container_runner_cuda.h 2024-08-08T20:48:50.1566971Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runner/pybind.h 2024-08-08T20:48:50.1567374Z creating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/ 2024-08-08T20:48:50.1569027Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/arrayref_tensor.h 2024-08-08T20:48:50.1569955Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/device_utils.h 2024-08-08T20:48:50.1571273Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/interface.h 2024-08-08T20:48:50.1574713Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/model.h 2024-08-08T20:48:50.1577296Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/model_container.h 2024-08-08T20:48:50.1578050Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/scalar_to_tensor.h 2024-08-08T20:48:50.1579046Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/thread_local.h 2024-08-08T20:48:50.1580273Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/utils.h 2024-08-08T20:48:50.1581750Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_runtime/utils_cuda.h 2024-08-08T20:48:50.1582284Z creating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/ 2024-08-08T20:48:50.1583471Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/mkldnn_tensor.h 2024-08-08T20:48:50.1584847Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/oss_proxy_executor.h 2024-08-08T20:48:50.1585708Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/proxy_executor.h 2024-08-08T20:48:50.1586480Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/tensor_converter.h 2024-08-08T20:48:50.1587414Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/utils.h 2024-08-08T20:48:50.1588138Z creating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/c/ 2024-08-08T20:48:50.1591357Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/c/shim.h 2024-08-08T20:48:50.1591830Z creating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/generated/ 2024-08-08T20:48:50.1594179Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/generated/c_shim_cpu.h 2024-08-08T20:48:50.1597038Z inflating: build/lib/torch/include/torch/csrc/inductor/aoti_torch/generated/c_shim_cuda.h 2024-08-08T20:48:50.1597714Z creating: build/lib/torch/include/torch/csrc/jit/ 2024-08-08T20:48:50.1598182Z inflating: build/lib/torch/include/torch/csrc/jit/jit_log.h 2024-08-08T20:48:50.1598780Z inflating: build/lib/torch/include/torch/csrc/jit/jit_opt_limit.h 2024-08-08T20:48:50.1599655Z inflating: build/lib/torch/include/torch/csrc/jit/resource_guard.h 2024-08-08T20:48:50.1600214Z creating: build/lib/torch/include/torch/csrc/jit/backends/ 2024-08-08T20:48:50.1601732Z inflating: build/lib/torch/include/torch/csrc/jit/backends/backend.h 2024-08-08T20:48:50.1603839Z inflating: build/lib/torch/include/torch/csrc/jit/backends/backend_debug_handler.h 2024-08-08T20:48:50.1604718Z inflating: build/lib/torch/include/torch/csrc/jit/backends/backend_debug_info.h 2024-08-08T20:48:50.1605427Z inflating: build/lib/torch/include/torch/csrc/jit/backends/backend_detail.h 2024-08-08T20:48:50.1606556Z inflating: build/lib/torch/include/torch/csrc/jit/backends/backend_exception.h 2024-08-08T20:48:50.1607048Z inflating: build/lib/torch/include/torch/csrc/jit/backends/backend_init.h 2024-08-08T20:48:50.1609474Z inflating: build/lib/torch/include/torch/csrc/jit/backends/backend_interface.h 2024-08-08T20:48:50.1610381Z inflating: build/lib/torch/include/torch/csrc/jit/backends/backend_preprocess.h 2024-08-08T20:48:50.1611167Z inflating: build/lib/torch/include/torch/csrc/jit/backends/backend_resolver.h 2024-08-08T20:48:50.1611766Z creating: build/lib/torch/include/torch/csrc/jit/passes/ 2024-08-08T20:48:50.1612492Z inflating: build/lib/torch/include/torch/csrc/jit/passes/add_if_then_else.h 2024-08-08T20:48:50.1613322Z inflating: build/lib/torch/include/torch/csrc/jit/passes/annotate_warns.h 2024-08-08T20:48:50.1613826Z inflating: build/lib/torch/include/torch/csrc/jit/passes/autocast.h 2024-08-08T20:48:50.1614452Z inflating: build/lib/torch/include/torch/csrc/jit/passes/bailout_graph.h 2024-08-08T20:48:50.1615219Z inflating: build/lib/torch/include/torch/csrc/jit/passes/batch_mm.h 2024-08-08T20:48:50.1616188Z inflating: build/lib/torch/include/torch/csrc/jit/passes/canonicalize.h 2024-08-08T20:48:50.1617292Z inflating: build/lib/torch/include/torch/csrc/jit/passes/canonicalize_graph_fuser_ops.h 2024-08-08T20:48:50.1618856Z inflating: build/lib/torch/include/torch/csrc/jit/passes/check_strict_fusion.h 2024-08-08T20:48:50.1619542Z inflating: build/lib/torch/include/torch/csrc/jit/passes/clear_profiling.h 2024-08-08T20:48:50.1620821Z inflating: build/lib/torch/include/torch/csrc/jit/passes/clear_undefinedness.h 2024-08-08T20:48:50.1621856Z inflating: build/lib/torch/include/torch/csrc/jit/passes/common_subexpression_elimination.h 2024-08-08T20:48:50.1622662Z inflating: build/lib/torch/include/torch/csrc/jit/passes/concat_opt.h 2024-08-08T20:48:50.1623320Z inflating: build/lib/torch/include/torch/csrc/jit/passes/constant_pooling.h 2024-08-08T20:48:50.1624479Z inflating: build/lib/torch/include/torch/csrc/jit/passes/constant_propagation.h 2024-08-08T20:48:50.1625699Z inflating: build/lib/torch/include/torch/csrc/jit/passes/create_autodiff_subgraphs.h 2024-08-08T20:48:50.1626585Z inflating: build/lib/torch/include/torch/csrc/jit/passes/create_functional_graphs.h 2024-08-08T20:48:50.1627248Z inflating: build/lib/torch/include/torch/csrc/jit/passes/dead_code_elimination.h 2024-08-08T20:48:50.1628010Z inflating: build/lib/torch/include/torch/csrc/jit/passes/decompose_ops.h 2024-08-08T20:48:50.1629293Z inflating: build/lib/torch/include/torch/csrc/jit/passes/device_type_analysis.h 2024-08-08T20:48:50.1629985Z inflating: build/lib/torch/include/torch/csrc/jit/passes/dtype_analysis.h 2024-08-08T20:48:50.1630854Z inflating: build/lib/torch/include/torch/csrc/jit/passes/eliminate_no_ops.h 2024-08-08T20:48:50.1631936Z inflating: build/lib/torch/include/torch/csrc/jit/passes/erase_number_types.h 2024-08-08T20:48:50.1633105Z inflating: build/lib/torch/include/torch/csrc/jit/passes/fixup_trace_scope_blocks.h 2024-08-08T20:48:50.1633852Z inflating: build/lib/torch/include/torch/csrc/jit/passes/fold_conv_bn.h 2024-08-08T20:48:50.1634789Z inflating: build/lib/torch/include/torch/csrc/jit/passes/fold_linear_bn.h 2024-08-08T20:48:50.1636004Z inflating: build/lib/torch/include/torch/csrc/jit/passes/freeze_module.h 2024-08-08T20:48:50.1636988Z inflating: build/lib/torch/include/torch/csrc/jit/passes/frozen_concat_linear.h 2024-08-08T20:48:50.1637790Z inflating: build/lib/torch/include/torch/csrc/jit/passes/frozen_conv_add_relu_fusion.h 2024-08-08T20:48:50.1638831Z inflating: build/lib/torch/include/torch/csrc/jit/passes/frozen_conv_folding.h 2024-08-08T20:48:50.1639975Z inflating: build/lib/torch/include/torch/csrc/jit/passes/frozen_graph_optimizations.h 2024-08-08T20:48:50.1640887Z inflating: build/lib/torch/include/torch/csrc/jit/passes/frozen_linear_folding.h 2024-08-08T20:48:50.1641800Z inflating: build/lib/torch/include/torch/csrc/jit/passes/frozen_linear_transpose.h 2024-08-08T20:48:50.1642714Z inflating: build/lib/torch/include/torch/csrc/jit/passes/frozen_ops_to_mkldnn.h 2024-08-08T20:48:50.1643553Z inflating: build/lib/torch/include/torch/csrc/jit/passes/fuse_linear.h 2024-08-08T20:48:50.1644105Z inflating: build/lib/torch/include/torch/csrc/jit/passes/fuse_relu.h 2024-08-08T20:48:50.1644820Z inflating: build/lib/torch/include/torch/csrc/jit/passes/graph_fuser.h 2024-08-08T20:48:50.1646066Z inflating: build/lib/torch/include/torch/csrc/jit/passes/graph_rewrite_helper.h 2024-08-08T20:48:50.1647217Z inflating: build/lib/torch/include/torch/csrc/jit/passes/guard_elimination.h 2024-08-08T20:48:50.1648136Z inflating: build/lib/torch/include/torch/csrc/jit/passes/hoist_conv_packed_params.h 2024-08-08T20:48:50.1649396Z inflating: build/lib/torch/include/torch/csrc/jit/passes/inline_autodiff_subgraphs.h 2024-08-08T20:48:50.1650032Z inflating: build/lib/torch/include/torch/csrc/jit/passes/inline_fork_wait.h 2024-08-08T20:48:50.1650661Z inflating: build/lib/torch/include/torch/csrc/jit/passes/inline_forked_closures.h 2024-08-08T20:48:50.1651525Z inflating: build/lib/torch/include/torch/csrc/jit/passes/inliner.h 2024-08-08T20:48:50.1652595Z inflating: build/lib/torch/include/torch/csrc/jit/passes/inplace_check.h 2024-08-08T20:48:50.1653812Z inflating: build/lib/torch/include/torch/csrc/jit/passes/insert_guards.h 2024-08-08T20:48:50.1654685Z inflating: build/lib/torch/include/torch/csrc/jit/passes/integer_value_refinement.h 2024-08-08T20:48:50.1655594Z inflating: build/lib/torch/include/torch/csrc/jit/passes/lift_closures.h 2024-08-08T20:48:50.1656379Z inflating: build/lib/torch/include/torch/csrc/jit/passes/liveness.h 2024-08-08T20:48:50.1657459Z inflating: build/lib/torch/include/torch/csrc/jit/passes/loop_unrolling.h 2024-08-08T20:48:50.1658300Z inflating: build/lib/torch/include/torch/csrc/jit/passes/lower_grad_of.h 2024-08-08T20:48:50.1659396Z inflating: build/lib/torch/include/torch/csrc/jit/passes/lower_graph.h 2024-08-08T20:48:50.1660775Z inflating: build/lib/torch/include/torch/csrc/jit/passes/lower_tuples.h 2024-08-08T20:48:50.1661653Z inflating: build/lib/torch/include/torch/csrc/jit/passes/metal_rewrite.h 2024-08-08T20:48:50.1662674Z inflating: build/lib/torch/include/torch/csrc/jit/passes/mkldnn_rewrite.h 2024-08-08T20:48:50.1663345Z inflating: build/lib/torch/include/torch/csrc/jit/passes/mobile_optimizer_type.h 2024-08-08T20:48:50.1664480Z inflating: build/lib/torch/include/torch/csrc/jit/passes/normalize_ops.h 2024-08-08T20:48:50.1665055Z inflating: build/lib/torch/include/torch/csrc/jit/passes/onednn_graph_fuser.h 2024-08-08T20:48:50.1665947Z inflating: build/lib/torch/include/torch/csrc/jit/passes/onnx.h 2024-08-08T20:48:50.1667712Z inflating: build/lib/torch/include/torch/csrc/jit/passes/pass_manager.h 2024-08-08T20:48:50.1668249Z inflating: build/lib/torch/include/torch/csrc/jit/passes/peephole.h 2024-08-08T20:48:50.1670263Z inflating: build/lib/torch/include/torch/csrc/jit/passes/peephole_alias_sensitive.h 2024-08-08T20:48:50.1671228Z inflating: build/lib/torch/include/torch/csrc/jit/passes/peephole_dict_idioms.h 2024-08-08T20:48:50.1672531Z inflating: build/lib/torch/include/torch/csrc/jit/passes/peephole_list_idioms.h 2024-08-08T20:48:50.1673338Z inflating: build/lib/torch/include/torch/csrc/jit/passes/peephole_non_tensor.h 2024-08-08T20:48:50.1674447Z inflating: build/lib/torch/include/torch/csrc/jit/passes/prepack_folding.h 2024-08-08T20:48:50.1675343Z inflating: build/lib/torch/include/torch/csrc/jit/passes/refine_tuple_types.h 2024-08-08T20:48:50.1676033Z inflating: build/lib/torch/include/torch/csrc/jit/passes/remove_dropout.h 2024-08-08T20:48:50.1677021Z inflating: build/lib/torch/include/torch/csrc/jit/passes/remove_exceptions.h 2024-08-08T20:48:50.1677660Z inflating: build/lib/torch/include/torch/csrc/jit/passes/remove_expands.h 2024-08-08T20:48:50.1678764Z inflating: build/lib/torch/include/torch/csrc/jit/passes/remove_inplace_ops.h 2024-08-08T20:48:50.1679850Z inflating: build/lib/torch/include/torch/csrc/jit/passes/remove_mutation.h 2024-08-08T20:48:50.1681391Z inflating: build/lib/torch/include/torch/csrc/jit/passes/remove_redundant_profiles.h 2024-08-08T20:48:50.1682413Z inflating: build/lib/torch/include/torch/csrc/jit/passes/replacement_of_old_operators.h 2024-08-08T20:48:50.1683252Z inflating: build/lib/torch/include/torch/csrc/jit/passes/requires_grad_analysis.h 2024-08-08T20:48:50.1684099Z inflating: build/lib/torch/include/torch/csrc/jit/passes/restore_mutation.h 2024-08-08T20:48:50.1684745Z inflating: build/lib/torch/include/torch/csrc/jit/passes/shape_analysis.h 2024-08-08T20:48:50.1685715Z inflating: build/lib/torch/include/torch/csrc/jit/passes/specialize_autogradzero.h 2024-08-08T20:48:50.1687177Z inflating: build/lib/torch/include/torch/csrc/jit/passes/subgraph_rewrite.h 2024-08-08T20:48:50.1688316Z inflating: build/lib/torch/include/torch/csrc/jit/passes/symbolic_shape_analysis.h 2024-08-08T20:48:50.1689307Z inflating: build/lib/torch/include/torch/csrc/jit/passes/symbolic_shape_cache.h 2024-08-08T20:48:50.1690556Z inflating: build/lib/torch/include/torch/csrc/jit/passes/symbolic_shape_runtime_fusion.h 2024-08-08T20:48:50.1691668Z inflating: build/lib/torch/include/torch/csrc/jit/passes/tensorexpr_fuser.h 2024-08-08T20:48:50.1692882Z inflating: build/lib/torch/include/torch/csrc/jit/passes/update_differentiable_graph_requires_grad.h 2024-08-08T20:48:50.1693917Z inflating: build/lib/torch/include/torch/csrc/jit/passes/value_refinement_utils.h 2024-08-08T20:48:50.1694568Z inflating: build/lib/torch/include/torch/csrc/jit/passes/variadic_ops.h 2024-08-08T20:48:50.1695163Z inflating: build/lib/torch/include/torch/csrc/jit/passes/vulkan_rewrite.h 2024-08-08T20:48:50.1696250Z inflating: build/lib/torch/include/torch/csrc/jit/passes/xnnpack_rewrite.h 2024-08-08T20:48:50.1697046Z creating: build/lib/torch/include/torch/csrc/jit/passes/quantization/ 2024-08-08T20:48:50.1698307Z inflating: build/lib/torch/include/torch/csrc/jit/passes/quantization/dedup_module_uses.h 2024-08-08T20:48:50.1699264Z inflating: build/lib/torch/include/torch/csrc/jit/passes/quantization/finalize.h 2024-08-08T20:48:50.1700328Z inflating: build/lib/torch/include/torch/csrc/jit/passes/quantization/fusion_passes.h 2024-08-08T20:48:50.1701827Z inflating: build/lib/torch/include/torch/csrc/jit/passes/quantization/helper.h 2024-08-08T20:48:50.1703044Z inflating: build/lib/torch/include/torch/csrc/jit/passes/quantization/insert_observers.h 2024-08-08T20:48:50.1704088Z inflating: build/lib/torch/include/torch/csrc/jit/passes/quantization/insert_quant_dequant.h 2024-08-08T20:48:50.1709010Z inflating: build/lib/torch/include/torch/csrc/jit/passes/quantization/quantization_patterns.h 2024-08-08T20:48:50.1710166Z inflating: build/lib/torch/include/torch/csrc/jit/passes/quantization/quantization_type.h 2024-08-08T20:48:50.1711302Z inflating: build/lib/torch/include/torch/csrc/jit/passes/quantization/register_packed_params.h 2024-08-08T20:48:50.1711954Z creating: build/lib/torch/include/torch/csrc/jit/passes/utils/ 2024-08-08T20:48:50.1712540Z inflating: build/lib/torch/include/torch/csrc/jit/passes/utils/check_alias_annotation.h 2024-08-08T20:48:50.1713687Z inflating: build/lib/torch/include/torch/csrc/jit/passes/utils/memory_dag.h 2024-08-08T20:48:50.1714775Z inflating: build/lib/torch/include/torch/csrc/jit/passes/utils/op_registry.h 2024-08-08T20:48:50.1715523Z inflating: build/lib/torch/include/torch/csrc/jit/passes/utils/optimization_utils.h 2024-08-08T20:48:50.1716783Z inflating: build/lib/torch/include/torch/csrc/jit/passes/utils/subgraph_utils.h 2024-08-08T20:48:50.1717252Z creating: build/lib/torch/include/torch/csrc/jit/runtime/ 2024-08-08T20:48:50.1719936Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/argument_spec.h 2024-08-08T20:48:50.1721141Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/autodiff.h 2024-08-08T20:48:50.1722382Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/calculate_necessary_args.h 2024-08-08T20:48:50.1723537Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/custom_operator.h 2024-08-08T20:48:50.1724421Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/decomposition_registry.h 2024-08-08T20:48:50.1725196Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/decomposition_registry_util.h 2024-08-08T20:48:50.1726166Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/exception_message.h 2024-08-08T20:48:50.1727551Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/graph_executor.h 2024-08-08T20:48:50.1728988Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/graph_executor_impl.h 2024-08-08T20:48:50.1730274Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/graph_iterator.h 2024-08-08T20:48:50.1731676Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/instruction.h 2024-08-08T20:48:50.1733129Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/interpreter.h 2024-08-08T20:48:50.1734150Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/jit_exception.h 2024-08-08T20:48:50.1734781Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/jit_trace.h 2024-08-08T20:48:50.1736006Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/logging.h 2024-08-08T20:48:50.1738284Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/operator.h 2024-08-08T20:48:50.1739207Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/operator_options.h 2024-08-08T20:48:50.1739845Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/print_handler.h 2024-08-08T20:48:50.1741030Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/profiling_graph_executor_impl.h 2024-08-08T20:48:50.1742657Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/profiling_record.h 2024-08-08T20:48:50.1746582Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/register_ops_utils.h 2024-08-08T20:48:50.1748135Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/script_profile.h 2024-08-08T20:48:50.1749412Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/serialized_shape_function_registry.h 2024-08-08T20:48:50.1750237Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/shape_function_registry.h 2024-08-08T20:48:50.1751189Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/simple_graph_executor_impl.h 2024-08-08T20:48:50.1752101Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/slice_indices_adjust.h 2024-08-08T20:48:50.1752738Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/symbolic_script.h 2024-08-08T20:48:50.1754129Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/symbolic_shape_registry.h 2024-08-08T20:48:50.1755011Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/symbolic_shape_registry_util.h 2024-08-08T20:48:50.1756105Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/vararg_functions.h 2024-08-08T20:48:50.1756854Z inflating: build/lib/torch/include/torch/csrc/jit/runtime/variable_tensor_list.h 2024-08-08T20:48:50.1757303Z creating: build/lib/torch/include/torch/csrc/jit/ir/ 2024-08-08T20:48:50.1760217Z inflating: build/lib/torch/include/torch/csrc/jit/ir/alias_analysis.h 2024-08-08T20:48:50.1761373Z inflating: build/lib/torch/include/torch/csrc/jit/ir/attributes.h 2024-08-08T20:48:50.1762348Z inflating: build/lib/torch/include/torch/csrc/jit/ir/constants.h 2024-08-08T20:48:50.1763909Z inflating: build/lib/torch/include/torch/csrc/jit/ir/graph_node_list.h 2024-08-08T20:48:50.1764690Z inflating: build/lib/torch/include/torch/csrc/jit/ir/graph_utils.h 2024-08-08T20:48:50.1770438Z inflating: build/lib/torch/include/torch/csrc/jit/ir/ir.h 2024-08-08T20:48:50.1771680Z inflating: build/lib/torch/include/torch/csrc/jit/ir/ir_views.h 2024-08-08T20:48:50.1772560Z inflating: build/lib/torch/include/torch/csrc/jit/ir/irparser.h 2024-08-08T20:48:50.1773618Z inflating: build/lib/torch/include/torch/csrc/jit/ir/named_value.h 2024-08-08T20:48:50.1775050Z inflating: build/lib/torch/include/torch/csrc/jit/ir/node_hashing.h 2024-08-08T20:48:50.1776490Z inflating: build/lib/torch/include/torch/csrc/jit/ir/scope.h 2024-08-08T20:48:50.1777847Z inflating: build/lib/torch/include/torch/csrc/jit/ir/subgraph_matcher.h 2024-08-08T20:48:50.1778787Z inflating: build/lib/torch/include/torch/csrc/jit/ir/type_hashing.h 2024-08-08T20:48:50.1779266Z creating: build/lib/torch/include/torch/csrc/jit/frontend/ 2024-08-08T20:48:50.1781609Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/source_range.h 2024-08-08T20:48:50.1784124Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/lexer.h 2024-08-08T20:48:50.1784924Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/strtod.h 2024-08-08T20:48:50.1785543Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/parser_constants.h 2024-08-08T20:48:50.1788454Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/function_schema_parser.h 2024-08-08T20:48:50.1789124Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/parse_string_literal.h 2024-08-08T20:48:50.1789947Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/schema_type_parser.h 2024-08-08T20:48:50.1791379Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/error_report.h 2024-08-08T20:48:50.1793022Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/tree.h 2024-08-08T20:48:50.1793690Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/builtin_functions.h 2024-08-08T20:48:50.1794565Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/canonicalize_modified_loop.h 2024-08-08T20:48:50.1796605Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/concrete_module_type.h 2024-08-08T20:48:50.1797767Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/convert_to_ssa.h 2024-08-08T20:48:50.1798677Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/edit_distance.h 2024-08-08T20:48:50.1799417Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/exit_transforms.h 2024-08-08T20:48:50.1800522Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/inline_loop_condition.h 2024-08-08T20:48:50.1801103Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/ir_emitter.h 2024-08-08T20:48:50.1802135Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/mini_environment.h 2024-08-08T20:48:50.1802988Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/name_mangler.h 2024-08-08T20:48:50.1803988Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/parser.h 2024-08-08T20:48:50.1805103Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/resolver.h 2024-08-08T20:48:50.1806235Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/schema_matching.h 2024-08-08T20:48:50.1807349Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/script_type_parser.h 2024-08-08T20:48:50.1808121Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/source_ref.h 2024-08-08T20:48:50.1811764Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/sugared_value.h 2024-08-08T20:48:50.1813731Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/tracer.h 2024-08-08T20:48:50.1817558Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/tree_views.h 2024-08-08T20:48:50.1818179Z inflating: build/lib/torch/include/torch/csrc/jit/frontend/versioned_symbols.h 2024-08-08T20:48:50.1818869Z creating: build/lib/torch/include/torch/csrc/jit/api/ 2024-08-08T20:48:50.1821106Z inflating: build/lib/torch/include/torch/csrc/jit/api/compilation_unit.h 2024-08-08T20:48:50.1822426Z inflating: build/lib/torch/include/torch/csrc/jit/api/function_impl.h 2024-08-08T20:48:50.1823452Z inflating: build/lib/torch/include/torch/csrc/jit/api/method.h 2024-08-08T20:48:50.1826566Z inflating: build/lib/torch/include/torch/csrc/jit/api/module.h 2024-08-08T20:48:50.1827934Z inflating: build/lib/torch/include/torch/csrc/jit/api/object.h 2024-08-08T20:48:50.1828615Z creating: build/lib/torch/include/torch/csrc/jit/serialization/ 2024-08-08T20:48:50.1830166Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/callstack_debug_info_serialization.h 2024-08-08T20:48:50.1830951Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/source_range_serialization.h 2024-08-08T20:48:50.1832701Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/export.h 2024-08-08T20:48:50.1833607Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/export_bytecode.h 2024-08-08T20:48:50.1834913Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/flatbuffer_serializer.h 2024-08-08T20:48:50.1835984Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/flatbuffer_serializer_jit.h 2024-08-08T20:48:50.1837208Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/import.h 2024-08-08T20:48:50.1838425Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/import_export_constants.h 2024-08-08T20:48:50.1839379Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/import_export_functions.h 2024-08-08T20:48:50.1840373Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/import_export_helpers.h 2024-08-08T20:48:50.1841056Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/import_read.h 2024-08-08T20:48:50.1842152Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/import_source.h 2024-08-08T20:48:50.1849825Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/mobile_bytecode_generated.h 2024-08-08T20:48:50.1850446Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/onnx.h 2024-08-08T20:48:50.1851853Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/pickle.h 2024-08-08T20:48:50.1854371Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/pickler.h 2024-08-08T20:48:50.1855108Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/python_print.h 2024-08-08T20:48:50.1856253Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/source_range_serialization_impl.h 2024-08-08T20:48:50.1857211Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/storage_context.h 2024-08-08T20:48:50.1858142Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/type_name_uniquer.h 2024-08-08T20:48:50.1860264Z inflating: build/lib/torch/include/torch/csrc/jit/serialization/unpickler.h 2024-08-08T20:48:50.1860798Z creating: build/lib/torch/include/torch/csrc/jit/python/ 2024-08-08T20:48:50.1861549Z inflating: build/lib/torch/include/torch/csrc/jit/python/init.h 2024-08-08T20:48:50.1862218Z inflating: build/lib/torch/include/torch/csrc/jit/python/module_python.h 2024-08-08T20:48:50.1863542Z inflating: build/lib/torch/include/torch/csrc/jit/python/pybind.h 2024-08-08T20:48:50.1868523Z inflating: build/lib/torch/include/torch/csrc/jit/python/pybind_utils.h 2024-08-08T20:48:50.1869667Z inflating: build/lib/torch/include/torch/csrc/jit/python/python_arg_flatten.h 2024-08-08T20:48:50.1870705Z inflating: build/lib/torch/include/torch/csrc/jit/python/python_custom_class.h 2024-08-08T20:48:50.1871751Z inflating: build/lib/torch/include/torch/csrc/jit/python/python_dict.h 2024-08-08T20:48:50.1872641Z inflating: build/lib/torch/include/torch/csrc/jit/python/python_ir.h 2024-08-08T20:48:50.1874287Z inflating: build/lib/torch/include/torch/csrc/jit/python/python_ivalue.h 2024-08-08T20:48:50.1875764Z inflating: build/lib/torch/include/torch/csrc/jit/python/python_list.h 2024-08-08T20:48:50.1878230Z inflating: build/lib/torch/include/torch/csrc/jit/python/python_sugared_value.h 2024-08-08T20:48:50.1879107Z inflating: build/lib/torch/include/torch/csrc/jit/python/python_tracer.h 2024-08-08T20:48:50.1879900Z inflating: build/lib/torch/include/torch/csrc/jit/python/python_tree_views.h 2024-08-08T20:48:50.1881050Z inflating: build/lib/torch/include/torch/csrc/jit/python/script_init.h 2024-08-08T20:48:50.1881887Z inflating: build/lib/torch/include/torch/csrc/jit/python/update_graph_executor_opt.h 2024-08-08T20:48:50.1882552Z inflating: build/lib/torch/include/torch/csrc/jit/python/utf8_decoding_ignore.h 2024-08-08T20:48:50.1882850Z creating: build/lib/torch/include/torch/csrc/jit/mobile/ 2024-08-08T20:48:50.1884347Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/code.h 2024-08-08T20:48:50.1885512Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/debug_info.h 2024-08-08T20:48:50.1887132Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/file_format.h 2024-08-08T20:48:50.1888581Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/flatbuffer_loader.h 2024-08-08T20:48:50.1889380Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/frame.h 2024-08-08T20:48:50.1890612Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/function.h 2024-08-08T20:48:50.1891736Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/import.h 2024-08-08T20:48:50.1893055Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/import_data.h 2024-08-08T20:48:50.1893881Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/import_export_common.h 2024-08-08T20:48:50.1894760Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/interpreter.h 2024-08-08T20:48:50.1895626Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/method.h 2024-08-08T20:48:50.1897492Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/module.h 2024-08-08T20:48:50.1898253Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/observer.h 2024-08-08T20:48:50.1899353Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/parse_bytecode.h 2024-08-08T20:48:50.1900516Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/parse_operators.h 2024-08-08T20:48:50.1901608Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/prim_ops_registery.h 2024-08-08T20:48:50.1903000Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/profiler_edge.h 2024-08-08T20:48:50.1903863Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/promoted_prim_ops.h 2024-08-08T20:48:50.1904837Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/quantization.h 2024-08-08T20:48:50.1906133Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/register_ops_common_utils.h 2024-08-08T20:48:50.1907007Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/type_parser.h 2024-08-08T20:48:50.1907985Z inflating: build/lib/torch/include/torch/csrc/jit/mobile/upgrader_mobile.h 2024-08-08T20:48:50.1908416Z creating: build/lib/torch/include/torch/csrc/jit/testing/ 2024-08-08T20:48:50.1909632Z inflating: build/lib/torch/include/torch/csrc/jit/testing/file_check.h 2024-08-08T20:48:50.1910659Z inflating: build/lib/torch/include/torch/csrc/jit/testing/hooks_for_testing.h 2024-08-08T20:48:50.1911045Z creating: build/lib/torch/include/torch/csrc/jit/tensorexpr/ 2024-08-08T20:48:50.1912632Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/analysis.h 2024-08-08T20:48:50.1913900Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/block_codegen.h 2024-08-08T20:48:50.1914932Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/bounds_inference.h 2024-08-08T20:48:50.1916288Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/bounds_overlap.h 2024-08-08T20:48:50.1917765Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/codegen.h 2024-08-08T20:48:50.1919047Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/cpp_codegen.h 2024-08-08T20:48:50.1919755Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/cpp_intrinsics.h 2024-08-08T20:48:50.1921759Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/cuda_codegen.h 2024-08-08T20:48:50.1922922Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/cuda_random.h 2024-08-08T20:48:50.1925004Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/eval.h 2024-08-08T20:48:50.1925688Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/exceptions.h 2024-08-08T20:48:50.1927852Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/expr.h 2024-08-08T20:48:50.1929031Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/external_functions.h 2024-08-08T20:48:50.1930192Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/external_functions_core.h 2024-08-08T20:48:50.1931323Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/external_functions_registry.h 2024-08-08T20:48:50.1932342Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/fwd_decls.h 2024-08-08T20:48:50.1933678Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/graph_opt.h 2024-08-08T20:48:50.1935143Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/half_support.h 2024-08-08T20:48:50.1936807Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/hash_provider.h 2024-08-08T20:48:50.1937545Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/intrinsic_symbols.h 2024-08-08T20:48:50.1940299Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/ir.h 2024-08-08T20:48:50.1940961Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/ir_cloner.h 2024-08-08T20:48:50.1942052Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/ir_mutator.h 2024-08-08T20:48:50.1943224Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/ir_printer.h 2024-08-08T20:48:50.1945710Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/ir_simplifier.h 2024-08-08T20:48:50.1946522Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/ir_verifier.h 2024-08-08T20:48:50.1947707Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/ir_visitor.h 2024-08-08T20:48:50.1949893Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/kernel.h 2024-08-08T20:48:50.1951042Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/llvm_codegen.h 2024-08-08T20:48:50.1952045Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/llvm_jit.h 2024-08-08T20:48:50.1955188Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/loopnest.h 2024-08-08T20:48:50.1956058Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/loopnest_randomization.h 2024-08-08T20:48:50.1956931Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/lowerings.h 2024-08-08T20:48:50.1959677Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/mem_dependency_checker.h 2024-08-08T20:48:50.1961742Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/reduction.h 2024-08-08T20:48:50.1963839Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/registerizer.h 2024-08-08T20:48:50.1966584Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/stmt.h 2024-08-08T20:48:50.1968160Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/tensor.h 2024-08-08T20:48:50.1968916Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/tensorexpr_init.h 2024-08-08T20:48:50.1970283Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/types.h 2024-08-08T20:48:50.1971294Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/unique_name_manager.h 2024-08-08T20:48:50.1972459Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/var_substitutor.h 2024-08-08T20:48:50.1973064Z creating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/ 2024-08-08T20:48:50.1974050Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/conv2d.h 2024-08-08T20:48:50.1974800Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/matmul.h 2024-08-08T20:48:50.1975879Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/misc.h 2024-08-08T20:48:50.1977032Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/norm.h 2024-08-08T20:48:50.1978270Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/operators.h 2024-08-08T20:48:50.1979307Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/pointwise.h 2024-08-08T20:48:50.1980556Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/quantization.h 2024-08-08T20:48:50.1981542Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/reduction.h 2024-08-08T20:48:50.1982290Z inflating: build/lib/torch/include/torch/csrc/jit/tensorexpr/operators/softmax.h 2024-08-08T20:48:50.1982706Z creating: build/lib/torch/include/torch/csrc/jit/codegen/ 2024-08-08T20:48:50.1983220Z creating: build/lib/torch/include/torch/csrc/jit/codegen/cuda/ 2024-08-08T20:48:50.1984153Z inflating: build/lib/torch/include/torch/csrc/jit/codegen/cuda/interface.h 2024-08-08T20:48:50.1984593Z creating: build/lib/torch/include/torch/csrc/onnx/ 2024-08-08T20:48:50.1985494Z inflating: build/lib/torch/include/torch/csrc/onnx/back_compat.h 2024-08-08T20:48:50.1985985Z inflating: build/lib/torch/include/torch/csrc/onnx/init.h 2024-08-08T20:48:50.1986668Z inflating: build/lib/torch/include/torch/csrc/onnx/onnx.h 2024-08-08T20:48:50.1987243Z creating: build/lib/torch/include/torch/csrc/profiler/ 2024-08-08T20:48:50.1988560Z inflating: build/lib/torch/include/torch/csrc/profiler/api.h 2024-08-08T20:48:50.1991513Z inflating: build/lib/torch/include/torch/csrc/profiler/collection.h 2024-08-08T20:48:50.1992137Z inflating: build/lib/torch/include/torch/csrc/profiler/combined_traceback.h 2024-08-08T20:48:50.1993696Z inflating: build/lib/torch/include/torch/csrc/profiler/containers.h 2024-08-08T20:48:50.1994549Z inflating: build/lib/torch/include/torch/csrc/profiler/data_flow.h 2024-08-08T20:48:50.1995489Z inflating: build/lib/torch/include/torch/csrc/profiler/events.h 2024-08-08T20:48:50.1996888Z inflating: build/lib/torch/include/torch/csrc/profiler/kineto_shim.h 2024-08-08T20:48:50.1997782Z inflating: build/lib/torch/include/torch/csrc/profiler/perf-inl.h 2024-08-08T20:48:50.1998745Z inflating: build/lib/torch/include/torch/csrc/profiler/perf.h 2024-08-08T20:48:50.2000382Z inflating: build/lib/torch/include/torch/csrc/profiler/util.h 2024-08-08T20:48:50.2000838Z creating: build/lib/torch/include/torch/csrc/profiler/orchestration/ 2024-08-08T20:48:50.2002753Z inflating: build/lib/torch/include/torch/csrc/profiler/orchestration/observer.h 2024-08-08T20:48:50.2003810Z inflating: build/lib/torch/include/torch/csrc/profiler/orchestration/python_tracer.h 2024-08-08T20:48:50.2005096Z inflating: build/lib/torch/include/torch/csrc/profiler/orchestration/vulkan.h 2024-08-08T20:48:50.2005761Z creating: build/lib/torch/include/torch/csrc/profiler/standalone/ 2024-08-08T20:48:50.2006860Z inflating: build/lib/torch/include/torch/csrc/profiler/standalone/execution_trace_observer.h 2024-08-08T20:48:50.2007824Z inflating: build/lib/torch/include/torch/csrc/profiler/standalone/itt_observer.h 2024-08-08T20:48:50.2008808Z inflating: build/lib/torch/include/torch/csrc/profiler/standalone/nvtx_observer.h 2024-08-08T20:48:50.2009593Z inflating: build/lib/torch/include/torch/csrc/profiler/standalone/privateuse1_observer.h 2024-08-08T20:48:50.2010203Z creating: build/lib/torch/include/torch/csrc/profiler/stubs/ 2024-08-08T20:48:50.2010719Z inflating: build/lib/torch/include/torch/csrc/profiler/stubs/base.h 2024-08-08T20:48:50.2011214Z creating: build/lib/torch/include/torch/csrc/profiler/unwind/ 2024-08-08T20:48:50.2012150Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/action.h 2024-08-08T20:48:50.2013207Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/communicate.h 2024-08-08T20:48:50.2014811Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/debug_info.h 2024-08-08T20:48:50.2015415Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/dwarf_enums.h 2024-08-08T20:48:50.2017019Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/dwarf_symbolize_enums.h 2024-08-08T20:48:50.2017932Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/eh_frame_hdr.h 2024-08-08T20:48:50.2019015Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/fast_symbolizer.h 2024-08-08T20:48:50.2021018Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/fde.h 2024-08-08T20:48:50.2021992Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/lexer.h 2024-08-08T20:48:50.2024480Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/line_number_program.h 2024-08-08T20:48:50.2025446Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/mem_file.h 2024-08-08T20:48:50.2026546Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/range_table.h 2024-08-08T20:48:50.2027755Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/sections.h 2024-08-08T20:48:50.2028851Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/unwind.h 2024-08-08T20:48:50.2030042Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/unwind_error.h 2024-08-08T20:48:50.2031348Z inflating: build/lib/torch/include/torch/csrc/profiler/unwind/unwinder.h 2024-08-08T20:48:50.2031910Z creating: build/lib/torch/include/torch/csrc/profiler/python/ 2024-08-08T20:48:50.2033101Z inflating: build/lib/torch/include/torch/csrc/profiler/python/combined_traceback.h 2024-08-08T20:48:50.2033811Z inflating: build/lib/torch/include/torch/csrc/profiler/python/init.h 2024-08-08T20:48:50.2034709Z inflating: build/lib/torch/include/torch/csrc/profiler/python/pybind.h 2024-08-08T20:48:50.2035160Z creating: build/lib/torch/include/torch/csrc/utils/ 2024-08-08T20:48:50.2036523Z inflating: build/lib/torch/include/torch/csrc/utils/byte_order.h 2024-08-08T20:48:50.2037304Z inflating: build/lib/torch/include/torch/csrc/utils/cpp_stacktraces.h 2024-08-08T20:48:50.2038083Z inflating: build/lib/torch/include/torch/csrc/utils/cuda_enabled.h 2024-08-08T20:48:50.2039216Z inflating: build/lib/torch/include/torch/csrc/utils/device_lazy_init.h 2024-08-08T20:48:50.2040406Z inflating: build/lib/torch/include/torch/csrc/utils/disable_torch_function.h 2024-08-08T20:48:50.2040737Z inflating: build/lib/torch/include/torch/csrc/utils/init.h 2024-08-08T20:48:50.2042206Z inflating: build/lib/torch/include/torch/csrc/utils/invalid_arguments.h 2024-08-08T20:48:50.2042601Z inflating: build/lib/torch/include/torch/csrc/utils/nested.h 2024-08-08T20:48:50.2044009Z inflating: build/lib/torch/include/torch/csrc/utils/numpy_stub.h 2024-08-08T20:48:50.2045116Z inflating: build/lib/torch/include/torch/csrc/utils/object_ptr.h 2024-08-08T20:48:50.2045883Z inflating: build/lib/torch/include/torch/csrc/utils/out_types.h 2024-08-08T20:48:50.2048109Z inflating: build/lib/torch/include/torch/csrc/utils/pybind.h 2024-08-08T20:48:50.2048853Z inflating: build/lib/torch/include/torch/csrc/utils/pycfunction_helpers.h 2024-08-08T20:48:50.2049736Z inflating: build/lib/torch/include/torch/csrc/utils/pyobject_preservation.h 2024-08-08T20:48:50.2054198Z inflating: build/lib/torch/include/torch/csrc/utils/python_arg_parser.h 2024-08-08T20:48:50.2054876Z inflating: build/lib/torch/include/torch/csrc/utils/python_compat.h 2024-08-08T20:48:50.2055726Z inflating: build/lib/torch/include/torch/csrc/utils/python_dispatch.h 2024-08-08T20:48:50.2056882Z inflating: build/lib/torch/include/torch/csrc/utils/python_numbers.h 2024-08-08T20:48:50.2059103Z inflating: build/lib/torch/include/torch/csrc/utils/python_raii.h 2024-08-08T20:48:50.2060289Z inflating: build/lib/torch/include/torch/csrc/utils/python_scalars.h 2024-08-08T20:48:50.2061701Z inflating: build/lib/torch/include/torch/csrc/utils/python_strings.h 2024-08-08T20:48:50.2062254Z inflating: build/lib/torch/include/torch/csrc/utils/python_stub.h 2024-08-08T20:48:50.2064325Z inflating: build/lib/torch/include/torch/csrc/utils/python_symnode.h 2024-08-08T20:48:50.2065452Z inflating: build/lib/torch/include/torch/csrc/utils/python_torch_function_mode.h 2024-08-08T20:48:50.2065976Z inflating: build/lib/torch/include/torch/csrc/utils/python_tuples.h 2024-08-08T20:48:50.2068599Z inflating: build/lib/torch/include/torch/csrc/utils/pythoncapi_compat.h 2024-08-08T20:48:50.2069467Z inflating: build/lib/torch/include/torch/csrc/utils/schema_info.h 2024-08-08T20:48:50.2070282Z inflating: build/lib/torch/include/torch/csrc/utils/six.h 2024-08-08T20:48:50.2071006Z inflating: build/lib/torch/include/torch/csrc/utils/structseq.h 2024-08-08T20:48:50.2072194Z inflating: build/lib/torch/include/torch/csrc/utils/tensor_apply.h 2024-08-08T20:48:50.2072843Z inflating: build/lib/torch/include/torch/csrc/utils/tensor_dtypes.h 2024-08-08T20:48:50.2073867Z inflating: build/lib/torch/include/torch/csrc/utils/tensor_flatten.h 2024-08-08T20:48:50.2074592Z extracting: build/lib/torch/include/torch/csrc/utils/tensor_layouts.h 2024-08-08T20:48:50.2075413Z inflating: build/lib/torch/include/torch/csrc/utils/tensor_list.h 2024-08-08T20:48:50.2076395Z inflating: build/lib/torch/include/torch/csrc/utils/tensor_memoryformats.h 2024-08-08T20:48:50.2077470Z inflating: build/lib/torch/include/torch/csrc/utils/tensor_new.h 2024-08-08T20:48:50.2078329Z inflating: build/lib/torch/include/torch/csrc/utils/tensor_numpy.h 2024-08-08T20:48:50.2079246Z inflating: build/lib/torch/include/torch/csrc/utils/tensor_qschemes.h 2024-08-08T20:48:50.2080358Z inflating: build/lib/torch/include/torch/csrc/utils/tensor_types.h 2024-08-08T20:48:50.2082469Z inflating: build/lib/torch/include/torch/csrc/utils/throughput_benchmark-inl.h 2024-08-08T20:48:50.2083428Z inflating: build/lib/torch/include/torch/csrc/utils/throughput_benchmark.h 2024-08-08T20:48:50.2084513Z inflating: build/lib/torch/include/torch/csrc/utils/torch_dispatch_mode.h 2024-08-08T20:48:50.2085612Z inflating: build/lib/torch/include/torch/csrc/utils/variadic.h 2024-08-08T20:48:50.2086357Z inflating: build/lib/torch/include/torch/csrc/utils/verbose.h 2024-08-08T20:48:50.2086929Z creating: build/lib/torch/include/torch/csrc/tensor/ 2024-08-08T20:48:50.2088415Z inflating: build/lib/torch/include/torch/csrc/tensor/python_tensor.h 2024-08-08T20:48:50.2088740Z creating: build/lib/torch/include/torch/csrc/lazy/ 2024-08-08T20:48:50.2089187Z creating: build/lib/torch/include/torch/csrc/lazy/backend/ 2024-08-08T20:48:50.2090304Z inflating: build/lib/torch/include/torch/csrc/lazy/backend/backend_data.h 2024-08-08T20:48:50.2091449Z inflating: build/lib/torch/include/torch/csrc/lazy/backend/backend_device.h 2024-08-08T20:48:50.2092898Z inflating: build/lib/torch/include/torch/csrc/lazy/backend/backend_interface.h 2024-08-08T20:48:50.2094138Z inflating: build/lib/torch/include/torch/csrc/lazy/backend/lowering_context.h 2024-08-08T20:48:50.2094739Z creating: build/lib/torch/include/torch/csrc/lazy/core/ 2024-08-08T20:48:50.2095710Z inflating: build/lib/torch/include/torch/csrc/lazy/core/cache.h 2024-08-08T20:48:50.2096496Z inflating: build/lib/torch/include/torch/csrc/lazy/core/config.h 2024-08-08T20:48:50.2097616Z inflating: build/lib/torch/include/torch/csrc/lazy/core/debug_util.h 2024-08-08T20:48:50.2098589Z inflating: build/lib/torch/include/torch/csrc/lazy/core/dynamic_ir.h 2024-08-08T20:48:50.2100154Z inflating: build/lib/torch/include/torch/csrc/lazy/core/hash.h 2024-08-08T20:48:50.2101235Z inflating: build/lib/torch/include/torch/csrc/lazy/core/helpers.h 2024-08-08T20:48:50.2103010Z inflating: build/lib/torch/include/torch/csrc/lazy/core/ir.h 2024-08-08T20:48:50.2104289Z inflating: build/lib/torch/include/torch/csrc/lazy/core/ir_builder.h 2024-08-08T20:48:50.2104957Z inflating: build/lib/torch/include/torch/csrc/lazy/core/ir_dump_util.h 2024-08-08T20:48:50.2106032Z inflating: build/lib/torch/include/torch/csrc/lazy/core/ir_metadata.h 2024-08-08T20:48:50.2107330Z inflating: build/lib/torch/include/torch/csrc/lazy/core/ir_util.h 2024-08-08T20:48:50.2109924Z inflating: build/lib/torch/include/torch/csrc/lazy/core/lazy_graph_executor.h 2024-08-08T20:48:50.2111094Z inflating: build/lib/torch/include/torch/csrc/lazy/core/metrics.h 2024-08-08T20:48:50.2113115Z inflating: build/lib/torch/include/torch/csrc/lazy/core/multi_wait.h 2024-08-08T20:48:50.2113851Z inflating: build/lib/torch/include/torch/csrc/lazy/core/permutation_util.h 2024-08-08T20:48:50.2114938Z inflating: build/lib/torch/include/torch/csrc/lazy/core/shape.h 2024-08-08T20:48:50.2117102Z inflating: build/lib/torch/include/torch/csrc/lazy/core/shape_inference.h 2024-08-08T20:48:50.2118956Z inflating: build/lib/torch/include/torch/csrc/lazy/core/tensor.h 2024-08-08T20:48:50.2119698Z inflating: build/lib/torch/include/torch/csrc/lazy/core/tensor_impl.h 2024-08-08T20:48:50.2121096Z inflating: build/lib/torch/include/torch/csrc/lazy/core/tensor_util.h 2024-08-08T20:48:50.2121947Z inflating: build/lib/torch/include/torch/csrc/lazy/core/thread_pool.h 2024-08-08T20:48:50.2123061Z inflating: build/lib/torch/include/torch/csrc/lazy/core/trie.h 2024-08-08T20:48:50.2124080Z inflating: build/lib/torch/include/torch/csrc/lazy/core/unique.h 2024-08-08T20:48:50.2125275Z inflating: build/lib/torch/include/torch/csrc/lazy/core/util.h 2024-08-08T20:48:50.2125864Z creating: build/lib/torch/include/torch/csrc/lazy/core/internal_ops/ 2024-08-08T20:48:50.2126975Z inflating: build/lib/torch/include/torch/csrc/lazy/core/internal_ops/ltc_ops.h 2024-08-08T20:48:50.2127525Z creating: build/lib/torch/include/torch/csrc/lazy/core/ops/ 2024-08-08T20:48:50.2128553Z inflating: build/lib/torch/include/torch/csrc/lazy/core/ops/arithmetic_ir_ops.h 2024-08-08T20:48:50.2129231Z inflating: build/lib/torch/include/torch/csrc/lazy/core/ops/utils.h 2024-08-08T20:48:50.2129740Z creating: build/lib/torch/include/torch/csrc/lazy/python/ 2024-08-08T20:48:50.2130743Z inflating: build/lib/torch/include/torch/csrc/lazy/python/python_util.h 2024-08-08T20:48:50.2131365Z creating: build/lib/torch/include/torch/csrc/lazy/ts_backend/ 2024-08-08T20:48:50.2132009Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/config.h 2024-08-08T20:48:50.2132893Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/dynamic_ir.h 2024-08-08T20:48:50.2133840Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/ir_builder.h 2024-08-08T20:48:50.2134893Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/tensor_aten_ops.h 2024-08-08T20:48:50.2135732Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/ts_autograd_functions.h 2024-08-08T20:48:50.2136748Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/ts_backend_impl.h 2024-08-08T20:48:50.2137545Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/ts_eager_fallback.h 2024-08-08T20:48:50.2139569Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/ts_lowering_context.h 2024-08-08T20:48:50.2140269Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/ts_node.h 2024-08-08T20:48:50.2141363Z inflating: build/lib/torch/include/torch/csrc/lazy/ts_backend/ts_node_lowering.h 2024-08-08T20:48:50.2143475Z creating: build/lib/torch/include/torch/csrc/xpu/ 2024-08-08T20:48:50.2144156Z inflating: build/lib/torch/include/torch/csrc/xpu/Event.h 2024-08-08T20:48:50.2145071Z inflating: build/lib/torch/include/torch/csrc/xpu/Module.h 2024-08-08T20:48:50.2145711Z inflating: build/lib/torch/include/torch/csrc/xpu/Stream.h 2024-08-08T20:48:50.2146092Z creating: build/lib/torch/include/pybind11/ 2024-08-08T20:48:50.2149300Z inflating: build/lib/torch/include/pybind11/attr.h 2024-08-08T20:48:50.2150988Z inflating: build/lib/torch/include/pybind11/buffer_info.h 2024-08-08T20:48:50.2157711Z inflating: build/lib/torch/include/pybind11/cast.h 2024-08-08T20:48:50.2159337Z inflating: build/lib/torch/include/pybind11/chrono.h 2024-08-08T20:48:50.2161549Z inflating: build/lib/torch/include/pybind11/common.h 2024-08-08T20:48:50.2162734Z inflating: build/lib/torch/include/pybind11/complex.h 2024-08-08T20:48:50.2163266Z inflating: build/lib/torch/include/pybind11/eigen.h 2024-08-08T20:48:50.2165316Z inflating: build/lib/torch/include/pybind11/embed.h 2024-08-08T20:48:50.2166394Z inflating: build/lib/torch/include/pybind11/eval.h 2024-08-08T20:48:50.2167764Z inflating: build/lib/torch/include/pybind11/functional.h 2024-08-08T20:48:50.2169661Z inflating: build/lib/torch/include/pybind11/gil.h 2024-08-08T20:48:50.2171082Z inflating: build/lib/torch/include/pybind11/gil_safe_call_once.h 2024-08-08T20:48:50.2172856Z inflating: build/lib/torch/include/pybind11/iostream.h 2024-08-08T20:48:50.2180559Z inflating: build/lib/torch/include/pybind11/numpy.h 2024-08-08T20:48:50.2182028Z inflating: build/lib/torch/include/pybind11/operators.h 2024-08-08T20:48:50.2183031Z inflating: build/lib/torch/include/pybind11/options.h 2024-08-08T20:48:50.2194742Z inflating: build/lib/torch/include/pybind11/pybind11.h 2024-08-08T20:48:50.2203401Z inflating: build/lib/torch/include/pybind11/pytypes.h 2024-08-08T20:48:50.2205457Z inflating: build/lib/torch/include/pybind11/stl.h 2024-08-08T20:48:50.2208665Z inflating: build/lib/torch/include/pybind11/stl_bind.h 2024-08-08T20:48:50.2209815Z inflating: build/lib/torch/include/pybind11/type_caster_pyobject_ptr.h 2024-08-08T20:48:50.2211115Z inflating: build/lib/torch/include/pybind11/typing.h 2024-08-08T20:48:50.2211536Z creating: build/lib/torch/include/pybind11/detail/ 2024-08-08T20:48:50.2215164Z inflating: build/lib/torch/include/pybind11/detail/class.h 2024-08-08T20:48:50.2220664Z inflating: build/lib/torch/include/pybind11/detail/common.h 2024-08-08T20:48:50.2221986Z inflating: build/lib/torch/include/pybind11/detail/descr.h 2024-08-08T20:48:50.2224496Z inflating: build/lib/torch/include/pybind11/detail/init.h 2024-08-08T20:48:50.2228194Z inflating: build/lib/torch/include/pybind11/detail/internals.h 2024-08-08T20:48:50.2233503Z inflating: build/lib/torch/include/pybind11/detail/type_caster_base.h 2024-08-08T20:48:50.2234116Z inflating: build/lib/torch/include/pybind11/detail/typeid.h 2024-08-08T20:48:50.2234548Z creating: build/lib/torch/include/pybind11/eigen/ 2024-08-08T20:48:50.2235512Z inflating: build/lib/torch/include/pybind11/eigen/common.h 2024-08-08T20:48:50.2239242Z inflating: build/lib/torch/include/pybind11/eigen/matrix.h 2024-08-08T20:48:50.2241510Z inflating: build/lib/torch/include/pybind11/eigen/tensor.h 2024-08-08T20:48:50.2241896Z creating: build/lib/torch/include/THC/ 2024-08-08T20:48:50.2242787Z inflating: build/lib/torch/include/THC/THCAtomics.cuh 2024-08-08T20:48:50.2243316Z extracting: build/lib/torch/include/THC/THCDeviceUtils.cuh 2024-08-08T20:48:50.2243798Z creating: build/lib/torch/include/tensorpipe/ 2024-08-08T20:48:50.2244487Z inflating: build/lib/torch/include/tensorpipe/tensorpipe.h 2024-08-08T20:48:50.2245550Z inflating: build/lib/torch/include/tensorpipe/config.h 2024-08-08T20:48:50.2246694Z inflating: build/lib/torch/include/tensorpipe/tensorpipe_cuda.h 2024-08-08T20:48:50.2247272Z inflating: build/lib/torch/include/tensorpipe/config_cuda.h 2024-08-08T20:48:50.2247754Z creating: build/lib/torch/include/tensorpipe/channel/ 2024-08-08T20:48:50.2249204Z inflating: build/lib/torch/include/tensorpipe/channel/context.h 2024-08-08T20:48:50.2249878Z inflating: build/lib/torch/include/tensorpipe/channel/error.h 2024-08-08T20:48:50.2250369Z creating: build/lib/torch/include/tensorpipe/channel/basic/ 2024-08-08T20:48:50.2251482Z inflating: build/lib/torch/include/tensorpipe/channel/basic/factory.h 2024-08-08T20:48:50.2252045Z creating: build/lib/torch/include/tensorpipe/channel/cma/ 2024-08-08T20:48:50.2252875Z inflating: build/lib/torch/include/tensorpipe/channel/cma/factory.h 2024-08-08T20:48:50.2253426Z creating: build/lib/torch/include/tensorpipe/channel/mpt/ 2024-08-08T20:48:50.2254086Z inflating: build/lib/torch/include/tensorpipe/channel/mpt/factory.h 2024-08-08T20:48:50.2254436Z creating: build/lib/torch/include/tensorpipe/channel/xth/ 2024-08-08T20:48:50.2255281Z inflating: build/lib/torch/include/tensorpipe/channel/xth/factory.h 2024-08-08T20:48:50.2255628Z creating: build/lib/torch/include/tensorpipe/common/ 2024-08-08T20:48:50.2256791Z inflating: build/lib/torch/include/tensorpipe/common/buffer.h 2024-08-08T20:48:50.2257320Z inflating: build/lib/torch/include/tensorpipe/common/cpu_buffer.h 2024-08-08T20:48:50.2258282Z inflating: build/lib/torch/include/tensorpipe/common/device.h 2024-08-08T20:48:50.2259707Z inflating: build/lib/torch/include/tensorpipe/common/error.h 2024-08-08T20:48:50.2262649Z inflating: build/lib/torch/include/tensorpipe/common/optional.h 2024-08-08T20:48:50.2263179Z inflating: build/lib/torch/include/tensorpipe/common/cuda_buffer.h 2024-08-08T20:48:50.2263867Z creating: build/lib/torch/include/tensorpipe/core/ 2024-08-08T20:48:50.2265094Z inflating: build/lib/torch/include/tensorpipe/core/context.h 2024-08-08T20:48:50.2265693Z inflating: build/lib/torch/include/tensorpipe/core/error.h 2024-08-08T20:48:50.2267039Z inflating: build/lib/torch/include/tensorpipe/core/listener.h 2024-08-08T20:48:50.2268124Z inflating: build/lib/torch/include/tensorpipe/core/message.h 2024-08-08T20:48:50.2269247Z inflating: build/lib/torch/include/tensorpipe/core/pipe.h 2024-08-08T20:48:50.2269649Z creating: build/lib/torch/include/tensorpipe/transport/ 2024-08-08T20:48:50.2271206Z inflating: build/lib/torch/include/tensorpipe/transport/context.h 2024-08-08T20:48:50.2272032Z inflating: build/lib/torch/include/tensorpipe/transport/error.h 2024-08-08T20:48:50.2272609Z creating: build/lib/torch/include/tensorpipe/transport/ibv/ 2024-08-08T20:48:50.2273479Z inflating: build/lib/torch/include/tensorpipe/transport/ibv/error.h 2024-08-08T20:48:50.2274146Z inflating: build/lib/torch/include/tensorpipe/transport/ibv/factory.h 2024-08-08T20:48:50.2275215Z inflating: build/lib/torch/include/tensorpipe/transport/ibv/utility.h 2024-08-08T20:48:50.2275627Z creating: build/lib/torch/include/tensorpipe/transport/shm/ 2024-08-08T20:48:50.2276600Z inflating: build/lib/torch/include/tensorpipe/transport/shm/factory.h 2024-08-08T20:48:50.2277110Z creating: build/lib/torch/include/tensorpipe/transport/uv/ 2024-08-08T20:48:50.2278005Z inflating: build/lib/torch/include/tensorpipe/transport/uv/error.h 2024-08-08T20:48:50.2278615Z inflating: build/lib/torch/include/tensorpipe/transport/uv/factory.h 2024-08-08T20:48:50.2279763Z inflating: build/lib/torch/include/tensorpipe/transport/uv/utility.h 2024-08-08T20:48:50.2280086Z creating: build/lib/torch/include/kineto/ 2024-08-08T20:48:50.2281261Z inflating: build/lib/torch/include/kineto/AbstractConfig.h 2024-08-08T20:48:50.2282799Z inflating: build/lib/torch/include/kineto/ActivityProfilerInterface.h 2024-08-08T20:48:50.2283390Z inflating: build/lib/torch/include/kineto/ActivityTraceInterface.h 2024-08-08T20:48:50.2284360Z inflating: build/lib/torch/include/kineto/ActivityType.h 2024-08-08T20:48:50.2286735Z inflating: build/lib/torch/include/kineto/Config.h 2024-08-08T20:48:50.2287285Z inflating: build/lib/torch/include/kineto/ClientInterface.h 2024-08-08T20:48:50.2288636Z inflating: build/lib/torch/include/kineto/GenericTraceActivity.h 2024-08-08T20:48:50.2290143Z inflating: build/lib/torch/include/kineto/IActivityProfiler.h 2024-08-08T20:48:50.2291138Z inflating: build/lib/torch/include/kineto/ILoggerObserver.h 2024-08-08T20:48:50.2292182Z inflating: build/lib/torch/include/kineto/ITraceActivity.h 2024-08-08T20:48:50.2292682Z inflating: build/lib/torch/include/kineto/LoggingAPI.h 2024-08-08T20:48:50.2293852Z inflating: build/lib/torch/include/kineto/TraceSpan.h 2024-08-08T20:48:50.2294473Z inflating: build/lib/torch/include/kineto/ThreadUtil.h 2024-08-08T20:48:50.2295569Z inflating: build/lib/torch/include/kineto/libkineto.h 2024-08-08T20:48:50.2296822Z inflating: build/lib/torch/include/kineto/time_since_epoch.h 2024-08-08T20:48:50.2297948Z inflating: build/lib/torch/include/kineto/output_base.h 2024-08-08T20:48:50.2298182Z creating: build/lib/torch/share/ 2024-08-08T20:48:50.2298476Z creating: build/lib/torch/share/cmake/ 2024-08-08T20:48:50.2298803Z creating: build/lib/torch/share/cmake/ATen/ 2024-08-08T20:48:50.2299655Z inflating: build/lib/torch/share/cmake/ATen/ATenConfig.cmake 2024-08-08T20:48:50.2300088Z creating: build/lib/torch/share/cmake/Caffe2/ 2024-08-08T20:48:50.2301986Z inflating: build/lib/torch/share/cmake/Caffe2/Caffe2Config.cmake 2024-08-08T20:48:50.2305988Z inflating: build/lib/torch/share/cmake/Caffe2/FindCUDAToolkit.cmake 2024-08-08T20:48:50.2306581Z inflating: build/lib/torch/share/cmake/Caffe2/FindCUSPARSELT.cmake 2024-08-08T20:48:50.2307680Z inflating: build/lib/torch/share/cmake/Caffe2/FindSYCLToolkit.cmake 2024-08-08T20:48:50.2309334Z inflating: build/lib/torch/share/cmake/Caffe2/Caffe2Targets.cmake 2024-08-08T20:48:50.2310170Z inflating: build/lib/torch/share/cmake/Caffe2/Caffe2Targets-release.cmake 2024-08-08T20:48:50.2310606Z creating: build/lib/torch/share/cmake/Caffe2/public/ 2024-08-08T20:48:50.2313009Z inflating: build/lib/torch/share/cmake/Caffe2/public/cuda.cmake 2024-08-08T20:48:50.2313681Z inflating: build/lib/torch/share/cmake/Caffe2/public/xpu.cmake 2024-08-08T20:48:50.2314839Z inflating: build/lib/torch/share/cmake/Caffe2/public/glog.cmake 2024-08-08T20:48:50.2315936Z inflating: build/lib/torch/share/cmake/Caffe2/public/gflags.cmake 2024-08-08T20:48:50.2317071Z inflating: build/lib/torch/share/cmake/Caffe2/public/mkl.cmake 2024-08-08T20:48:50.2318058Z inflating: build/lib/torch/share/cmake/Caffe2/public/mkldnn.cmake 2024-08-08T20:48:50.2319237Z inflating: build/lib/torch/share/cmake/Caffe2/public/protobuf.cmake 2024-08-08T20:48:50.2321954Z inflating: build/lib/torch/share/cmake/Caffe2/public/utils.cmake 2024-08-08T20:48:50.2323519Z inflating: build/lib/torch/share/cmake/Caffe2/public/LoadHIP.cmake 2024-08-08T20:48:50.2324130Z creating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/ 2024-08-08T20:48:50.2324891Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/FindCUDA.cmake 2024-08-08T20:48:50.2326069Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/FindCUDNN.cmake 2024-08-08T20:48:50.2326698Z creating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/ 2024-08-08T20:48:50.2327772Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/CMakeInitializeConfigs.cmake 2024-08-08T20:48:50.2336000Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindCUDA.cmake 2024-08-08T20:48:50.2338668Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindPackageHandleStandardArgs.cmake 2024-08-08T20:48:50.2339702Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindPackageMessage.cmake 2024-08-08T20:48:50.2340534Z creating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindCUDA/ 2024-08-08T20:48:50.2341431Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindCUDA/make2cmake.cmake 2024-08-08T20:48:50.2342488Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindCUDA/parse_cubin.cmake 2024-08-08T20:48:50.2344546Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindCUDA/run_nvcc.cmake 2024-08-08T20:48:50.2346519Z inflating: build/lib/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindCUDA/select_compute_arch.cmake 2024-08-08T20:48:50.2346801Z creating: build/lib/torch/share/cmake/Tensorpipe/ 2024-08-08T20:48:50.2348213Z inflating: build/lib/torch/share/cmake/Tensorpipe/TensorpipeTargets.cmake 2024-08-08T20:48:50.2349412Z inflating: build/lib/torch/share/cmake/Tensorpipe/TensorpipeTargets-release.cmake 2024-08-08T20:48:50.2350017Z creating: build/lib/torch/share/cmake/Torch/ 2024-08-08T20:48:50.2350512Z inflating: build/lib/torch/share/cmake/Torch/TorchConfigVersion.cmake 2024-08-08T20:48:50.2351569Z inflating: build/lib/torch/share/cmake/Torch/TorchConfig.cmake 2024-08-08T20:48:50.2351830Z creating: build/lib/torch/lib/ 2024-08-08T20:48:50.2422236Z inflating: build/lib/torch/lib/libc10_cuda.so 2024-08-08T20:48:50.2537149Z inflating: build/lib/torch/lib/libc10.so 2024-08-08T20:48:50.2539373Z inflating: build/lib/torch/lib/libcaffe2_nvrtc.so 2024-08-08T20:48:50.2626421Z inflating: build/lib/torch/lib/libtorchbind_test.so 2024-08-08T20:48:50.2651636Z inflating: build/lib/torch/lib/libjitbackend_test.so 2024-08-08T20:48:50.2682938Z inflating: build/lib/torch/lib/libbackend_with_compiler.so 2024-08-08T20:48:50.2714669Z inflating: build/lib/torch/lib/libaoti_custom_ops.so 2024-08-08T20:48:50.2717898Z inflating: build/lib/torch/lib/libc10d_cuda_test.so 2024-08-08T20:48:53.1957582Z inflating: build/lib/torch/lib/libtorch_cpu.so 2024-08-08T20:48:56.7248003Z inflating: build/lib/torch/lib/libtorch_cuda.so 2024-08-08T20:48:56.7249687Z inflating: build/lib/torch/lib/libtorch.so 2024-08-08T20:48:56.7251971Z inflating: build/lib/torch/lib/libtorch_global_deps.so 2024-08-08T20:48:56.7253098Z creating: build/lib/torchgen/ 2024-08-08T20:48:56.7254026Z inflating: build/lib/torchgen/__init__.py 2024-08-08T20:48:56.7255070Z inflating: build/lib/torchgen/code_template.py 2024-08-08T20:48:56.7255703Z inflating: build/lib/torchgen/context.py 2024-08-08T20:48:56.7265476Z inflating: build/lib/torchgen/gen.py 2024-08-08T20:48:56.7267742Z inflating: build/lib/torchgen/gen_aoti_c_shim.py 2024-08-08T20:48:56.7270458Z inflating: build/lib/torchgen/gen_backend_stubs.py 2024-08-08T20:48:56.7274431Z inflating: build/lib/torchgen/gen_executorch.py 2024-08-08T20:48:56.7278703Z inflating: build/lib/torchgen/gen_functionalization_type.py 2024-08-08T20:48:56.7281435Z inflating: build/lib/torchgen/gen_lazy_tensor.py 2024-08-08T20:48:56.7282731Z inflating: build/lib/torchgen/gen_vmap_plumbing.py 2024-08-08T20:48:56.7283799Z inflating: build/lib/torchgen/local.py 2024-08-08T20:48:56.7294343Z inflating: build/lib/torchgen/model.py 2024-08-08T20:48:56.7297859Z inflating: build/lib/torchgen/native_function_generation.py 2024-08-08T20:48:56.7300159Z inflating: build/lib/torchgen/utils.py 2024-08-08T20:48:56.7301221Z inflating: build/lib/torchgen/yaml_utils.py 2024-08-08T20:48:56.7302038Z creating: build/lib/torchgen/aoti/ 2024-08-08T20:48:56.7302887Z extracting: build/lib/torchgen/aoti/__init__.py 2024-08-08T20:48:56.7303837Z inflating: build/lib/torchgen/aoti/fallback_ops.py 2024-08-08T20:48:56.7304846Z creating: build/lib/torchgen/api/ 2024-08-08T20:48:56.7305789Z extracting: build/lib/torchgen/api/__init__.py 2024-08-08T20:48:56.7309209Z inflating: build/lib/torchgen/api/autograd.py 2024-08-08T20:48:56.7311382Z inflating: build/lib/torchgen/api/cpp.py 2024-08-08T20:48:56.7312433Z inflating: build/lib/torchgen/api/dispatcher.py 2024-08-08T20:48:56.7313931Z inflating: build/lib/torchgen/api/functionalization.py 2024-08-08T20:48:56.7316376Z inflating: build/lib/torchgen/api/lazy.py 2024-08-08T20:48:56.7317270Z inflating: build/lib/torchgen/api/meta.py 2024-08-08T20:48:56.7318237Z inflating: build/lib/torchgen/api/native.py 2024-08-08T20:48:56.7324051Z inflating: build/lib/torchgen/api/python.py 2024-08-08T20:48:56.7325238Z inflating: build/lib/torchgen/api/structured.py 2024-08-08T20:48:56.7328316Z inflating: build/lib/torchgen/api/translate.py 2024-08-08T20:48:56.7329708Z inflating: build/lib/torchgen/api/ufunc.py 2024-08-08T20:48:56.7331394Z inflating: build/lib/torchgen/api/unboxing.py 2024-08-08T20:48:56.7332470Z creating: build/lib/torchgen/api/types/ 2024-08-08T20:48:56.7333184Z inflating: build/lib/torchgen/api/types/__init__.py 2024-08-08T20:48:56.7334820Z inflating: build/lib/torchgen/api/types/signatures.py 2024-08-08T20:48:56.7336421Z inflating: build/lib/torchgen/api/types/types.py 2024-08-08T20:48:56.7337849Z inflating: build/lib/torchgen/api/types/types_base.py 2024-08-08T20:48:56.7338982Z creating: build/lib/torchgen/dest/ 2024-08-08T20:48:56.7339578Z inflating: build/lib/torchgen/dest/__init__.py 2024-08-08T20:48:56.7342404Z inflating: build/lib/torchgen/dest/lazy_ir.py 2024-08-08T20:48:56.7343570Z inflating: build/lib/torchgen/dest/lazy_ts_lowering.py 2024-08-08T20:48:56.7344544Z inflating: build/lib/torchgen/dest/native_functions.py 2024-08-08T20:48:56.7348543Z inflating: build/lib/torchgen/dest/register_dispatch_key.py 2024-08-08T20:48:56.7350871Z inflating: build/lib/torchgen/dest/ufunc.py 2024-08-08T20:48:56.7351740Z creating: build/lib/torchgen/executorch/ 2024-08-08T20:48:56.7352502Z extracting: build/lib/torchgen/executorch/__init__.py 2024-08-08T20:48:56.7353533Z inflating: build/lib/torchgen/executorch/model.py 2024-08-08T20:48:56.7354712Z inflating: build/lib/torchgen/executorch/parse.py 2024-08-08T20:48:56.7355618Z creating: build/lib/torchgen/executorch/api/ 2024-08-08T20:48:56.7356737Z extracting: build/lib/torchgen/executorch/api/__init__.py 2024-08-08T20:48:56.7357547Z inflating: build/lib/torchgen/executorch/api/custom_ops.py 2024-08-08T20:48:56.7359152Z inflating: build/lib/torchgen/executorch/api/et_cpp.py 2024-08-08T20:48:56.7360917Z inflating: build/lib/torchgen/executorch/api/unboxing.py 2024-08-08T20:48:56.7362137Z creating: build/lib/torchgen/executorch/api/types/ 2024-08-08T20:48:56.7363312Z inflating: build/lib/torchgen/executorch/api/types/__init__.py 2024-08-08T20:48:56.7364816Z inflating: build/lib/torchgen/executorch/api/types/signatures.py 2024-08-08T20:48:56.7365892Z inflating: build/lib/torchgen/executorch/api/types/types.py 2024-08-08T20:48:56.7367038Z creating: build/lib/torchgen/operator_versions/ 2024-08-08T20:48:56.7368239Z extracting: build/lib/torchgen/operator_versions/__init__.py 2024-08-08T20:48:56.7369417Z inflating: build/lib/torchgen/operator_versions/gen_mobile_upgraders.py 2024-08-08T20:48:56.7370856Z inflating: build/lib/torchgen/operator_versions/gen_mobile_upgraders_constant.py 2024-08-08T20:48:56.7372149Z creating: build/lib/torchgen/selective_build/ 2024-08-08T20:48:56.7373201Z extracting: build/lib/torchgen/selective_build/__init__.py 2024-08-08T20:48:56.7374006Z inflating: build/lib/torchgen/selective_build/operator.py 2024-08-08T20:48:56.7375172Z inflating: build/lib/torchgen/selective_build/selector.py 2024-08-08T20:48:56.7376258Z creating: build/lib/torchgen/static_runtime/ 2024-08-08T20:48:56.7376977Z extracting: build/lib/torchgen/static_runtime/__init__.py 2024-08-08T20:48:56.7377747Z inflating: build/lib/torchgen/static_runtime/config.py 2024-08-08T20:48:56.7379240Z inflating: build/lib/torchgen/static_runtime/gen_static_runtime_ops.py 2024-08-08T20:48:56.7380161Z inflating: build/lib/torchgen/static_runtime/generator.py 2024-08-08T20:48:56.7404543Z inflating: build/lib/libfmt.a 2024-08-08T20:48:56.7406760Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-08T20:48:56.7411752Z inflating: build/lib/libshm.so 2024-08-08T20:48:56.8024678Z inflating: build/lib/libkineto.a 2024-08-08T20:48:57.0417115Z inflating: build/lib/libtorch_python.so 2024-08-08T20:48:57.0459333Z inflating: build/lib/libnnapi_backend.so 2024-08-08T20:48:57.0459938Z creating: build/bin/ 2024-08-08T20:48:57.1004481Z inflating: build/bin/protoc-3.13.0.0 2024-08-08T20:48:57.1548231Z inflating: build/bin/protoc 2024-08-08T20:48:57.1964439Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-08T20:48:57.2363560Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-08T20:48:57.2798024Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-08T20:48:57.2866217Z inflating: build/bin/TCPStoreTest 2024-08-08T20:48:57.2930824Z inflating: build/bin/FileStoreTest 2024-08-08T20:48:57.2996047Z inflating: build/bin/BackoffTest 2024-08-08T20:48:57.3061305Z inflating: build/bin/HashStoreTest 2024-08-08T20:48:57.3077899Z inflating: build/bin/ProcessGroupMPITest 2024-08-08T20:48:57.3143836Z inflating: build/bin/test_edge_op_registration 2024-08-08T20:48:57.3147359Z inflating: build/bin/example_allreduce 2024-08-08T20:48:57.3211419Z inflating: build/bin/Dimname_test 2024-08-08T20:48:57.3290421Z inflating: build/bin/MaybeOwned_test 2024-08-08T20:48:57.3360468Z inflating: build/bin/NamedTensor_test 2024-08-08T20:48:57.3449758Z inflating: build/bin/Dict_test 2024-08-08T20:48:57.3522469Z inflating: build/bin/apply_utils_test 2024-08-08T20:48:57.3594712Z inflating: build/bin/atest 2024-08-08T20:48:57.3672007Z inflating: build/bin/basic 2024-08-08T20:48:57.3739062Z inflating: build/bin/broadcast_test 2024-08-08T20:48:57.3801718Z inflating: build/bin/cpu_allocator_test 2024-08-08T20:48:57.3873080Z inflating: build/bin/cpu_generator_test 2024-08-08T20:48:57.3937735Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-08T20:48:57.3999376Z inflating: build/bin/dispatch_key_set_test 2024-08-08T20:48:57.4061238Z inflating: build/bin/dlconvertor_test 2024-08-08T20:48:57.4174782Z inflating: build/bin/cpu_rng_test 2024-08-08T20:48:57.4245748Z inflating: build/bin/extension_backend_test 2024-08-08T20:48:57.4313672Z inflating: build/bin/half_test 2024-08-08T20:48:57.4430482Z inflating: build/bin/ivalue_test 2024-08-08T20:48:57.4491834Z inflating: build/bin/lazy_tensor_test 2024-08-08T20:48:57.4558108Z inflating: build/bin/math_kernel_test 2024-08-08T20:48:57.4624610Z inflating: build/bin/memory_format_test 2024-08-08T20:48:57.4689892Z inflating: build/bin/memory_overlapping_test 2024-08-08T20:48:57.4754581Z inflating: build/bin/mobile_memory_cleanup 2024-08-08T20:48:57.4823233Z inflating: build/bin/native_test 2024-08-08T20:48:57.4885944Z inflating: build/bin/operator_name_test 2024-08-08T20:48:57.4948983Z inflating: build/bin/operators_test 2024-08-08T20:48:57.5013278Z inflating: build/bin/packedtensoraccessor_test 2024-08-08T20:48:57.5095507Z inflating: build/bin/pow_test 2024-08-08T20:48:57.5165750Z inflating: build/bin/quantized_test 2024-08-08T20:48:57.5226617Z inflating: build/bin/reduce_ops_test 2024-08-08T20:48:57.5289729Z inflating: build/bin/reportMemoryUsage_test 2024-08-08T20:48:57.5358655Z inflating: build/bin/scalar_tensor_test 2024-08-08T20:48:57.5430268Z inflating: build/bin/scalar_test 2024-08-08T20:48:57.5494231Z inflating: build/bin/StorageUtils_test 2024-08-08T20:48:57.5558217Z inflating: build/bin/stride_properties_test 2024-08-08T20:48:57.5625146Z inflating: build/bin/test_parallel 2024-08-08T20:48:57.5721123Z inflating: build/bin/tensor_iterator_test 2024-08-08T20:48:57.5789610Z inflating: build/bin/type_ptr_test 2024-08-08T20:48:57.5793341Z inflating: build/bin/thread_init_test 2024-08-08T20:48:57.5867010Z inflating: build/bin/type_test 2024-08-08T20:48:57.5930682Z inflating: build/bin/undefined_tensor_test 2024-08-08T20:48:57.5932750Z inflating: build/bin/verify_api_visibility 2024-08-08T20:48:57.5995988Z inflating: build/bin/weakref_test 2024-08-08T20:48:57.6080662Z inflating: build/bin/legacy_vmap_test 2024-08-08T20:48:57.6143375Z inflating: build/bin/wrapdim_test 2024-08-08T20:48:57.6206992Z inflating: build/bin/xla_tensor_test 2024-08-08T20:48:57.6280366Z inflating: build/bin/IListRef_test 2024-08-08T20:48:57.6409434Z inflating: build/bin/List_test 2024-08-08T20:48:57.6557507Z inflating: build/bin/kernel_function_legacy_test 2024-08-08T20:48:57.6675243Z inflating: build/bin/kernel_function_test 2024-08-08T20:48:57.6756058Z inflating: build/bin/KernelFunction_test 2024-08-08T20:48:57.6912114Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-08T20:48:57.7037628Z inflating: build/bin/kernel_lambda_test 2024-08-08T20:48:57.7112799Z inflating: build/bin/kernel_stackbased_test 2024-08-08T20:48:57.7175115Z inflating: build/bin/CppSignature_test 2024-08-08T20:48:57.7292378Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-08T20:48:57.7351773Z inflating: build/bin/op_allowlist_test 2024-08-08T20:48:57.7420328Z inflating: build/bin/backend_fallback_test 2024-08-08T20:48:57.7785152Z inflating: build/bin/op_registration_test 2024-08-08T20:48:57.7862923Z inflating: build/bin/inline_container_test 2024-08-08T20:48:57.7927350Z inflating: build/bin/cuda_apply_test 2024-08-08T20:48:57.7991688Z inflating: build/bin/cuda_allocator_test 2024-08-08T20:48:57.8058288Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-08T20:48:57.8130525Z inflating: build/bin/cuda_atomic_ops_test 2024-08-08T20:48:57.8216219Z inflating: build/bin/cuda_complex_math_test 2024-08-08T20:48:57.8277172Z inflating: build/bin/cuda_device_test 2024-08-08T20:48:57.8349413Z inflating: build/bin/cuda_complex_test 2024-08-08T20:48:57.8421290Z inflating: build/bin/cuda_cub_test 2024-08-08T20:48:57.8484102Z inflating: build/bin/cuda_dlconvertor_test 2024-08-08T20:48:57.8546619Z inflating: build/bin/cuda_integer_divider_test 2024-08-08T20:48:57.8627284Z inflating: build/bin/cuda_distributions_test 2024-08-08T20:48:57.8696635Z inflating: build/bin/cuda_generator_test 2024-08-08T20:48:57.8757261Z inflating: build/bin/cuda_half_test 2024-08-08T20:48:57.8818810Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-08T20:48:57.8879654Z inflating: build/bin/cuda_optional_test 2024-08-08T20:48:57.8943616Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-08T20:48:57.9018129Z inflating: build/bin/cuda_stream_test 2024-08-08T20:48:57.9081654Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-08T20:48:57.9141747Z inflating: build/bin/cuda_cudnn_test 2024-08-08T20:48:57.9205658Z inflating: build/bin/cuda_vectorized_test 2024-08-08T20:48:57.9223426Z inflating: build/bin/tutorial_tensorexpr 2024-08-08T20:48:57.9303862Z inflating: build/bin/ProcessGroupGlooTest 2024-08-08T20:48:57.9374344Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-08T20:48:57.9451712Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-08T20:48:57.9528072Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-08T20:48:57.9595719Z inflating: build/bin/test_dist_autograd 2024-08-08T20:48:57.9598870Z inflating: build/bin/parallel_benchmark 2024-08-08T20:48:57.9683261Z inflating: build/bin/test_cpp_rpc 2024-08-08T20:48:57.9765446Z inflating: build/bin/test_mobile_nnc 2024-08-08T20:48:57.9776420Z inflating: build/bin/aot_model_compiler_test 2024-08-08T20:48:58.0782919Z inflating: build/bin/test_tensorexpr 2024-08-08T20:48:58.1201196Z inflating: build/bin/test_lazy 2024-08-08T20:48:58.2632644Z inflating: build/bin/test_api 2024-08-08T20:48:58.3337200Z inflating: build/bin/test_jit 2024-08-08T20:48:58.3396877Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-08T20:48:58.3458069Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-08T20:48:58.3521012Z inflating: build/bin/c10_DeviceGuard_test 2024-08-08T20:48:58.3582734Z inflating: build/bin/c10_Device_test 2024-08-08T20:48:58.3654498Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-08T20:48:58.3719874Z inflating: build/bin/c10_Scalar_test 2024-08-08T20:48:58.3780077Z inflating: build/bin/c10_StreamGuard_test 2024-08-08T20:48:58.3842055Z inflating: build/bin/c10_SymInt_test 2024-08-08T20:48:58.3909095Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-08T20:48:58.3977481Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-08T20:48:58.4045805Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-08T20:48:58.4132293Z inflating: build/bin/c10_cow_test 2024-08-08T20:48:58.4197297Z inflating: build/bin/c10_Bitset_test 2024-08-08T20:48:58.4256735Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-08T20:48:58.4318128Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-08T20:48:58.4386851Z inflating: build/bin/c10_LeftRight_test 2024-08-08T20:48:58.4448836Z inflating: build/bin/c10_Half_test 2024-08-08T20:48:58.4516286Z inflating: build/bin/c10_Metaprogramming_test 2024-08-08T20:48:58.4577383Z inflating: build/bin/c10_Synchronized_test 2024-08-08T20:48:58.4645119Z inflating: build/bin/c10_ThreadLocal_test 2024-08-08T20:48:58.4708210Z inflating: build/bin/c10_TypeIndex_test 2024-08-08T20:48:58.4770834Z inflating: build/bin/c10_TypeList_test 2024-08-08T20:48:58.4830440Z inflating: build/bin/c10_TypeTraits_test 2024-08-08T20:48:58.4893995Z inflating: build/bin/c10_accumulate_test 2024-08-08T20:48:58.4961711Z inflating: build/bin/c10_bfloat16_test 2024-08-08T20:48:58.5023247Z inflating: build/bin/c10_bit_cast_test 2024-08-08T20:48:58.5092182Z inflating: build/bin/c10_complex_math_test 2024-08-08T20:48:58.5160006Z inflating: build/bin/c10_complex_test 2024-08-08T20:48:58.5224265Z inflating: build/bin/c10_exception_test 2024-08-08T20:48:58.5285996Z inflating: build/bin/c10_flags_test 2024-08-08T20:48:58.5347380Z inflating: build/bin/c10_generic_math_test 2024-08-08T20:48:58.5545852Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-08T20:48:58.5608304Z inflating: build/bin/c10_irange_test 2024-08-08T20:48:58.5674139Z inflating: build/bin/c10_lazy_test 2024-08-08T20:48:58.5743522Z inflating: build/bin/c10_logging_test 2024-08-08T20:48:58.5809919Z inflating: build/bin/c10_registry_test 2024-08-08T20:48:58.5993353Z inflating: build/bin/c10_small_vector_test 2024-08-08T20:48:58.6069843Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-08T20:48:58.6161227Z inflating: build/bin/c10_optional_test 2024-08-08T20:48:58.6223936Z inflating: build/bin/c10_ssize_test 2024-08-08T20:48:58.6296147Z inflating: build/bin/c10_string_view_test 2024-08-08T20:48:58.6359329Z inflating: build/bin/c10_string_util_test 2024-08-08T20:48:58.6421077Z inflating: build/bin/c10_tempfile_test 2024-08-08T20:48:58.6489869Z inflating: build/bin/c10_typeid_test 2024-08-08T20:48:58.6640914Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-08T20:48:58.6792282Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-08T20:48:58.6942430Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-08T20:48:58.7093181Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-08T20:48:58.7243642Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-08T20:48:58.7303681Z inflating: build/bin/c10_cuda_CUDATest 2024-08-08T20:48:58.7454749Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-08T20:48:58.7605632Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-08T20:48:58.7610637Z inflating: build/bin/torch_shm_manager 2024-08-08T20:48:58.7611530Z creating: .additional_ci_files/ 2024-08-08T20:48:58.7683488Z inflating: .additional_ci_files/test-times.json 2024-08-08T20:48:58.7971054Z inflating: .additional_ci_files/test-class-times.json 2024-08-08T20:48:58.8042922Z ##[group]Run rm artifacts.zip 2024-08-08T20:48:58.8043395Z rm artifacts.zip 2024-08-08T20:48:58.8051608Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:48:58.8052234Z env: 2024-08-08T20:48:58.8052553Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:58.8053115Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:58.8053721Z ##[endgroup] 2024-08-08T20:48:58.8986826Z ##[group]Run df -H 2024-08-08T20:48:58.8987218Z df -H 2024-08-08T20:48:58.8994285Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:48:58.8994937Z env: 2024-08-08T20:48:58.8995277Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:58.8996012Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:58.8996648Z ##[endgroup] 2024-08-08T20:48:58.9046549Z Filesystem Size Used Avail Use% Mounted on 2024-08-08T20:48:58.9047874Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-08T20:48:58.9048472Z tmpfs 65G 0 65G 0% /dev/shm 2024-08-08T20:48:58.9049419Z tmpfs 26G 562k 26G 1% /run 2024-08-08T20:48:58.9050227Z /dev/xvda1 161G 39G 123G 24% / 2024-08-08T20:48:58.9050753Z tmpfs 65G 13k 65G 1% /tmp 2024-08-08T20:48:58.9051220Z /dev/xvda128 11M 1.4M 9.2M 13% /boot/efi 2024-08-08T20:48:58.9051831Z tmpfs 13G 0 13G 0% /run/user/0 2024-08-08T20:48:58.9102607Z Prepare all required actions 2024-08-08T20:48:58.9103209Z Getting action download info 2024-08-08T20:48:59.0528649Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-08T20:48:59.0529234Z with: 2024-08-08T20:48:59.0529548Z env: 2024-08-08T20:48:59.0529878Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:59.0530421Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:59.0531021Z ##[endgroup] 2024-08-08T20:48:59.0573237Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-08T20:48:59.0573797Z with: 2024-08-08T20:48:59.0574104Z name: td_results 2024-08-08T20:48:59.0574491Z s3-bucket: gha-artifacts 2024-08-08T20:48:59.0574922Z region: us-east-1 2024-08-08T20:48:59.0575270Z env: 2024-08-08T20:48:59.0575603Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:59.0576169Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:59.0576754Z ##[endgroup] 2024-08-08T20:48:59.6709899Z (node:79924) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-08T20:48:59.6710865Z 2024-08-08T20:48:59.6711191Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-08T20:48:59.6712121Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-08T20:48:59.6713242Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-08T20:48:59.7701119Z Found 1 objects with prefix pytorch/pytorch/10308807531/td_results/ 2024-08-08T20:48:59.7703412Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-08T20:48:59.8405132Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-08T20:48:59.8415814Z Artifact download has finished successfully 2024-08-08T20:48:59.8622480Z ##[group]Run mkdir -p .additional_ci_files 2024-08-08T20:48:59.8623086Z mkdir -p .additional_ci_files 2024-08-08T20:48:59.8623807Z mv td_results.json .additional_ci_files/td_results.json 2024-08-08T20:48:59.8631837Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:48:59.8632465Z env: 2024-08-08T20:48:59.8632804Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:59.8633586Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:59.8634194Z ##[endgroup] 2024-08-08T20:48:59.8730953Z ##[group]Run .github/scripts/parse_ref.py 2024-08-08T20:48:59.8731585Z .github/scripts/parse_ref.py 2024-08-08T20:48:59.8738197Z shell: /usr/bin/bash -e {0} 2024-08-08T20:48:59.8738637Z env: 2024-08-08T20:48:59.8738971Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:59.8739516Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:59.8740121Z ##[endgroup] 2024-08-08T20:48:59.9101475Z Prepare all required actions 2024-08-08T20:48:59.9714310Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-08T20:48:59.9714883Z with: 2024-08-08T20:48:59.9715598Z github-token: *** 2024-08-08T20:48:59.9715961Z env: 2024-08-08T20:48:59.9716287Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:59.9716845Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:59.9717428Z ##[endgroup] 2024-08-08T20:48:59.9738330Z ##[group]Run set -eux 2024-08-08T20:48:59.9738745Z set -eux 2024-08-08T20:48:59.9739475Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-08T20:48:59.9746449Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:48:59.9747054Z env: 2024-08-08T20:48:59.9747393Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:48:59.9747954Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:48:59.9748748Z GITHUB_TOKEN: *** 2024-08-08T20:48:59.9749136Z ##[endgroup] 2024-08-08T20:48:59.9778123Z + python3 .github/scripts/get_workflow_job_id.py 10308807531 i-09af04530f81e3933 2024-08-08T20:49:01.1204232Z setting job-id=28538244287 2024-08-08T20:49:01.1205712Z setting job-name=linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build-test / test (default, 4, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-08T20:49:01.1489463Z Prepare all required actions 2024-08-08T20:49:01.1490055Z Getting action download info 2024-08-08T20:49:01.2898778Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-08T20:49:01.2899349Z with: 2024-08-08T20:49:01.2899920Z github-token: *** 2024-08-08T20:49:01.2904287Z test-matrix: {"include": [{"config": "nogpu_AVX512", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "nogpu_NO_AVX2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}]} 2024-08-08T20:49:01.2909428Z job-name: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build-test / test (default, 4, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-08T20:49:01.2910606Z env: 2024-08-08T20:49:01.2910944Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:49:01.2911500Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:49:01.2912086Z ##[endgroup] 2024-08-08T20:49:01.2986420Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-08T20:49:01.2987446Z with: 2024-08-08T20:49:01.2987751Z shell: bash 2024-08-08T20:49:01.2988116Z timeout_minutes: 10 2024-08-08T20:49:01.2988495Z max_attempts: 5 2024-08-08T20:49:01.2988932Z retry_wait_seconds: 30 2024-08-08T20:49:01.2990362Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-08T20:49:01.2991846Z polling_interval_seconds: 1 2024-08-08T20:49:01.2992305Z warning_on_retry: true 2024-08-08T20:49:01.2993004Z continue_on_error: false 2024-08-08T20:49:01.2993398Z env: 2024-08-08T20:49:01.2993723Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:49:01.2994278Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:49:01.2995088Z GITHUB_TOKEN: *** 2024-08-08T20:49:01.2995451Z ##[endgroup] 2024-08-08T20:49:01.3923360Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-08T20:49:01.7244049Z Defaulting to user installation because normal site-packages is not writeable 2024-08-08T20:49:01.7451223Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2024-08-08T20:49:01.7457455Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2024-08-08T20:49:01.7618121Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-08T20:49:01.7632888Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2024-08-08T20:49:01.7645387Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-08T20:49:01.7653823Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-08T20:49:02.3723396Z Command completed after 1 attempt(s). 2024-08-08T20:49:02.3783350Z ##[group]Run set -x 2024-08-08T20:49:02.3783762Z set -x 2024-08-08T20:49:02.3784121Z  2024-08-08T20:49:02.3784800Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-08T20:49:02.3785631Z # in runner workspace 2024-08-08T20:49:02.3786283Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-08T20:49:02.3794547Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:49:02.3795198Z env: 2024-08-08T20:49:02.3795534Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:49:02.3796095Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:49:02.3796693Z ##[endgroup] 2024-08-08T20:49:02.3828079Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-08T20:49:02.4148025Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-08T20:49:02.4148704Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-08T20:49:02.4149279Z echo "Job name: ${JOB_NAME}" 2024-08-08T20:49:02.4149751Z  2024-08-08T20:49:02.4150428Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-08T20:49:02.4151281Z # in runner workspace 2024-08-08T20:49:02.4152001Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-08T20:49:02.4152800Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-08T20:49:02.4153368Z  --job-name "${JOB_NAME}" \ 2024-08-08T20:49:02.4157871Z  --test-matrix "{"include": [{"config": "nogpu_AVX512", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "nogpu_NO_AVX2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}]}" \ 2024-08-08T20:49:02.4162806Z  --selected-test-configs "" \ 2024-08-08T20:49:02.4163380Z  --pr-number "${PR_NUMBER}" \ 2024-08-08T20:49:02.4163880Z  --tag "${TAG}" \ 2024-08-08T20:49:02.4164608Z  --event-name "${EVENT_NAME}" \ 2024-08-08T20:49:02.4165154Z  --schedule "${SCHEDULE}" \ 2024-08-08T20:49:02.4165664Z  --branch "${HEAD_BRANCH}" 2024-08-08T20:49:02.4172540Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:49:02.4173170Z env: 2024-08-08T20:49:02.4173490Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:49:02.4174047Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:49:02.4174892Z GITHUB_TOKEN: *** 2024-08-08T20:49:02.4175995Z JOB_NAME: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build-test / test (default, 4, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-08T20:49:02.4177168Z PR_NUMBER: 2024-08-08T20:49:02.4177538Z TAG: ciflow/trunk/132710 2024-08-08T20:49:02.4177966Z EVENT_NAME: push 2024-08-08T20:49:02.4178322Z SCHEDULE: 2024-08-08T20:49:02.4178684Z HEAD_BRANCH: 2024-08-08T20:49:02.4179047Z ##[endgroup] 2024-08-08T20:49:02.4206821Z Workflow: trunk 2024-08-08T20:49:02.4208501Z Job name: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build-test / test (default, 4, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-08T20:49:02.7124818Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-08T20:49:02.9902480Z ##[group]Run echo "Filtered matrix:" 2024-08-08T20:49:02.9903047Z echo "Filtered matrix:" 2024-08-08T20:49:02.9907650Z echo "{"include": [{"config": "nogpu_AVX512", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "nogpu_NO_AVX2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}]}" 2024-08-08T20:49:02.9912075Z  2024-08-08T20:49:02.9912407Z echo 2024-08-08T20:49:02.9912853Z echo "Is the current job unstable? False" 2024-08-08T20:49:02.9913422Z  2024-08-08T20:49:02.9913747Z echo 2024-08-08T20:49:02.9914165Z echo "Is keep-going label set? False" 2024-08-08T20:49:02.9914707Z  2024-08-08T20:49:02.9915033Z echo 2024-08-08T20:49:02.9915398Z echo "Renabled issues? " 2024-08-08T20:49:02.9922607Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:49:02.9923238Z env: 2024-08-08T20:49:02.9923562Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:49:02.9924134Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:49:02.9924741Z ##[endgroup] 2024-08-08T20:49:02.9954371Z Filtered matrix: 2024-08-08T20:49:02.9960857Z {include: [{config: nogpu_AVX512, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: nogpu_NO_AVX2, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: jit_legacy, shard: 1, num_shards: 1, runner: amz2023.linux.4xlarge.nvidia.gpu}, {config: default, shard: 1, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}, {config: default, shard: 2, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}, {config: default, shard: 3, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}, {config: default, shard: 4, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}, {config: default, shard: 5, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}]} 2024-08-08T20:49:02.9965008Z 2024-08-08T20:49:02.9965212Z Is the current job unstable? False 2024-08-08T20:49:02.9965582Z 2024-08-08T20:49:02.9966057Z Is keep-going label set? False 2024-08-08T20:49:02.9966383Z 2024-08-08T20:49:02.9966547Z Renabled issues? 2024-08-08T20:49:03.0044131Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-08T20:49:03.0045067Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-08T20:49:03.0051666Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T20:49:03.0052277Z env: 2024-08-08T20:49:03.0052619Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:49:03.0053184Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:49:03.0053773Z JOB_TIMEOUT: 240 2024-08-08T20:49:03.0054151Z ##[endgroup] 2024-08-08T20:49:03.0150605Z ##[group]Run set -x 2024-08-08T20:49:03.0151152Z set -x 2024-08-08T20:49:03.0151517Z  2024-08-08T20:49:03.0151949Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-08T20:49:03.0152619Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-08T20:49:03.0153333Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-08T20:49:03.0153968Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-08T20:49:03.0154462Z else 2024-08-08T20:49:03.0154901Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-08T20:49:03.0155422Z fi 2024-08-08T20:49:03.0155740Z  2024-08-08T20:49:03.0156328Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-08T20:49:03.0157295Z # TODO: Stop building test binaries as part of the build phase 2024-08-08T20:49:03.0158132Z # Used for GPU_FLAG since that doesn't play nice 2024-08-08T20:49:03.0159165Z # shellcheck disable=SC2086,SC2090 2024-08-08T20:49:03.0159733Z container_name=$(docker run \ 2024-08-08T20:49:03.0160258Z  ${GPU_FLAG:-} \ 2024-08-08T20:49:03.0160720Z  -e BUILD_ENVIRONMENT \ 2024-08-08T20:49:03.0161200Z  -e PR_NUMBER \ 2024-08-08T20:49:03.0161651Z  -e GITHUB_ACTIONS \ 2024-08-08T20:49:03.0162134Z  -e GITHUB_REPOSITORY \ 2024-08-08T20:49:03.0162620Z  -e GITHUB_WORKFLOW \ 2024-08-08T20:49:03.0163100Z  -e GITHUB_JOB \ 2024-08-08T20:49:03.0163557Z  -e GITHUB_RUN_ID \ 2024-08-08T20:49:03.0164039Z  -e GITHUB_RUN_NUMBER \ 2024-08-08T20:49:03.0164524Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-08T20:49:03.0165016Z  -e JOB_ID \ 2024-08-08T20:49:03.0165447Z  -e JOB_NAME \ 2024-08-08T20:49:03.0165860Z  -e BASE_SHA \ 2024-08-08T20:49:03.0166283Z  -e BRANCH \ 2024-08-08T20:49:03.0166697Z  -e SHA1 \ 2024-08-08T20:49:03.0167108Z  -e AWS_DEFAULT_REGION \ 2024-08-08T20:49:03.0167611Z  -e IN_WHEEL_TEST \ 2024-08-08T20:49:03.0168079Z  -e SHARD_NUMBER \ 2024-08-08T20:49:03.0168522Z  -e TEST_CONFIG \ 2024-08-08T20:49:03.0168975Z  -e NUM_TEST_SHARDS \ 2024-08-08T20:49:03.0169463Z  -e REENABLED_ISSUES \ 2024-08-08T20:49:03.0169955Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-08T20:49:03.0170483Z  -e VERBOSE_TEST_LOGS \ 2024-08-08T20:49:03.0170989Z  -e TEST_SHOWLOCALS \ 2024-08-08T20:49:03.0171466Z  -e NO_TEST_TIMEOUT \ 2024-08-08T20:49:03.0171933Z  -e NO_TD \ 2024-08-08T20:49:03.0172352Z  -e TD_DISTRIBUTED \ 2024-08-08T20:49:03.0172805Z  -e PR_LABELS \ 2024-08-08T20:49:03.0173307Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-08T20:49:03.0173875Z  -e SCCACHE_BUCKET \ 2024-08-08T20:49:03.0174352Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-08T20:49:03.0174864Z  -e XLA_CUDA \ 2024-08-08T20:49:03.0175373Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-08T20:49:03.0175994Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-08T20:49:03.0176650Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-08T20:49:03.0177294Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-08T20:49:03.0177858Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-08T20:49:03.0178383Z  -e DASHBOARD_TAG \ 2024-08-08T20:49:03.0178978Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-08T20:49:03.0179871Z  --security-opt seccomp=unconfined \ 2024-08-08T20:49:03.0180453Z  --cap-add=SYS_PTRACE \ 2024-08-08T20:49:03.0180940Z  --ipc=host \ 2024-08-08T20:49:03.0181386Z  --shm-size="${SHM_SIZE}" \ 2024-08-08T20:49:03.0181862Z  --tty \ 2024-08-08T20:49:03.0182249Z  --detach \ 2024-08-08T20:49:03.0182693Z  --name="${container_name}" \ 2024-08-08T20:49:03.0183358Z  --user jenkins \ 2024-08-08T20:49:03.0183996Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-08T20:49:03.0184700Z  -w /var/lib/jenkins/workspace \ 2024-08-08T20:49:03.0185226Z  "${DOCKER_IMAGE}" 2024-08-08T20:49:03.0185659Z ) 2024-08-08T20:49:03.0186160Z # Propagate download.pytorch.org IP to container 2024-08-08T20:49:03.0187310Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-08T20:49:03.0188559Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-08T20:49:03.0189777Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-08T20:49:03.0196803Z shell: /usr/bin/bash -e {0} 2024-08-08T20:49:03.0197250Z env: 2024-08-08T20:49:03.0197589Z GIT_DEFAULT_BRANCH: main 2024-08-08T20:49:03.0198137Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:49:03.0199080Z BUILD_ENVIRONMENT: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build 2024-08-08T20:49:03.0199880Z PR_NUMBER: 2024-08-08T20:49:03.0200275Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-08T20:49:03.0200772Z GITHUB_WORKFLOW: trunk 2024-08-08T20:49:03.0201189Z GITHUB_JOB: test 2024-08-08T20:49:03.0201576Z GITHUB_RUN_ID: 10308807531 2024-08-08T20:49:03.0202005Z GITHUB_RUN_NUMBER: 90547 2024-08-08T20:49:03.0202439Z GITHUB_RUN_ATTEMPT: 1 2024-08-08T20:49:03.0202848Z JOB_ID: 28538244287 2024-08-08T20:49:03.0203953Z JOB_NAME: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build-test / test (default, 4, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-08T20:49:03.0205144Z BRANCH: 2024-08-08T20:49:03.0205556Z SHA1: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:03.0206179Z BASE_SHA: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:03.0206758Z TEST_CONFIG: default 2024-08-08T20:49:03.0207161Z SHARD_NUMBER: 4 2024-08-08T20:49:03.0207527Z NUM_TEST_SHARDS: 5 2024-08-08T20:49:03.0207928Z REENABLED_ISSUES: 2024-08-08T20:49:03.0208343Z CONTINUE_THROUGH_ERROR: False 2024-08-08T20:49:03.0208801Z VERBOSE_TEST_LOGS: False 2024-08-08T20:49:03.0209237Z TEST_SHOWLOCALS: False 2024-08-08T20:49:03.0209662Z NO_TEST_TIMEOUT: False 2024-08-08T20:49:03.0210055Z NO_TD: False 2024-08-08T20:49:03.0210427Z TD_DISTRIBUTED: False 2024-08-08T20:49:03.0210946Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-08T20:49:03.0211542Z SCCACHE_S3_KEY_PREFIX: trunk 2024-08-08T20:49:03.0212004Z SHM_SIZE: 2g 2024-08-08T20:49:03.0213227Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:49:03.0214547Z XLA_CUDA: 2024-08-08T20:49:03.0215145Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-08T20:49:03.0215920Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-08T20:49:03.0216465Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-08T20:49:03.0216955Z DASHBOARD_TAG: 2024-08-08T20:49:03.0217343Z HUGGING_FACE_HUB_TOKEN: 2024-08-08T20:49:03.0217769Z ##[endgroup] 2024-08-08T20:49:03.0245279Z + [[ default == \m\u\l\t\i\g\p\u ]] 2024-08-08T20:49:03.0246297Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *onnx* ]] 2024-08-08T20:49:03.0247131Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-08T20:49:03.0255253Z +++ nproc --ignore=2 2024-08-08T20:49:03.0295268Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10308807531 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T20:49:21.9165702Z + container_name=eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T20:49:21.9168116Z + grep download.pytorch.org /etc/hosts 2024-08-08T20:49:21.9170320Z + docker exec -i eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-08T20:49:22.0913572Z + echo DOCKER_CONTAINER_ID=eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T20:49:22.0918889Z ++ echo dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl dist/torch_no_python-2.5.0a0+gitb9d86fa-py3-none-any.whl 2024-08-08T20:49:22.0921752Z + docker exec -t eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 sh -c 'pip install dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl dist/torch_no_python-2.5.0a0+gitb9d86fa-py3-none-any.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-08T20:49:22.6377693Z Processing ./dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl 2024-08-08T20:49:22.9691759Z Processing ./dist/torch_no_python-2.5.0a0+gitb9d86fa-py3-none-any.whl (from torch-no-python==2.5.0a0+gitb9d86fa) 2024-08-08T20:49:24.0147381Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa) (3.13.1) 2024-08-08T20:49:24.0156653Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa) (4.12.2) 2024-08-08T20:49:24.0161865Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa) (2.8.8) 2024-08-08T20:49:24.0167788Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa) (3.1.4) 2024-08-08T20:49:24.0172989Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa) (2024.6.1) 2024-08-08T20:49:24.0190538Z Requirement already satisfied: sympy>=1.13.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa) (1.13.1) 2024-08-08T20:49:24.0314219Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch-no-python==2.5.0a0+gitb9d86fa->torch-no-python==2.5.0a0+gitb9d86fa) (3.3.0) 2024-08-08T20:49:24.0400354Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch-no-python==2.5.0a0+gitb9d86fa->torch-no-python==2.5.0a0+gitb9d86fa) (1.21.2) 2024-08-08T20:49:24.0449666Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.0->torch==2.5.0a0+gitb9d86fa) (1.3.0) 2024-08-08T20:49:24.1745581Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb9d86fa) (2.1.5) 2024-08-08T20:49:25.4269324Z Installing collected packages: torch-no-python, torch 2024-08-08T20:49:42.6765661Z Successfully installed torch-2.5.0a0+gitb9d86fa torch-no-python-2.5.0a0+gitb9d86fa 2024-08-08T20:49:42.7500757Z ++ dirname .ci/pytorch/test.sh 2024-08-08T20:49:42.7508695Z + source .ci/pytorch/common.sh 2024-08-08T20:49:42.7512241Z +++ dirname .ci/pytorch/common.sh 2024-08-08T20:49:42.7521128Z ++ source .ci/pytorch/common_utils.sh 2024-08-08T20:49:42.7523077Z +++ declare -f -t trap_add 2024-08-08T20:49:42.7531065Z ++ set -ex 2024-08-08T20:49:42.7532276Z ++ [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *rocm* ]] 2024-08-08T20:49:42.7533235Z ++ BUILD_TEST_LIBTORCH=0 2024-08-08T20:49:42.7534218Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build != *rocm* ]] 2024-08-08T20:49:42.7535857Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-08T20:49:42.7552518Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-08T20:49:42.7553302Z + trap_add cleanup_workspace EXIT 2024-08-08T20:49:42.7553821Z + trap_add_cmd=cleanup_workspace 2024-08-08T20:49:42.7554284Z + shift 2024-08-08T20:49:42.7554627Z + for trap_add_name in "$@" 2024-08-08T20:49:42.7562751Z +++ trap -p EXIT 2024-08-08T20:49:42.7566067Z ++ eval 'extract_trap_cmd ' 2024-08-08T20:49:42.7566603Z +++ extract_trap_cmd 2024-08-08T20:49:42.7567063Z +++ printf '%s\n' '' 2024-08-08T20:49:42.7567469Z ++ printf '%s\n' cleanup_workspace 2024-08-08T20:49:42.7569650Z + trap -- ' 2024-08-08T20:49:42.7570084Z cleanup_workspace' EXIT 2024-08-08T20:49:42.7570692Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-08T20:49:43.4210111Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-08T20:49:43.4231576Z + echo 'Environment variables:' 2024-08-08T20:49:43.4232038Z Environment variables: 2024-08-08T20:49:43.4232356Z + env 2024-08-08T20:49:43.4242139Z INSTALLED_DB=yes 2024-08-08T20:49:43.4243102Z NV_LIBCUBLAS_VERSION=12.4.2.65-1 2024-08-08T20:49:43.4243633Z NVIDIA_VISIBLE_DEVICES=all 2024-08-08T20:49:43.4244480Z NV_NVML_DEV_VERSION=12.4.99-1 2024-08-08T20:49:43.4245301Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-08T20:49:43.4246053Z CONTINUE_THROUGH_ERROR=False 2024-08-08T20:49:43.4246685Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.20.5-1+cuda12.4 2024-08-08T20:49:43.4247325Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.20.5-1 2024-08-08T20:49:43.4248241Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build 2024-08-08T20:49:43.4249047Z HOSTNAME=eca2856b64b6 2024-08-08T20:49:43.4250169Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4251435Z GITHUB_ACTION=__self 2024-08-08T20:49:43.4251876Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-08T20:49:43.4259875Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536 2024-08-08T20:49:43.4269402Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.2.65-1 2024-08-08T20:49:43.4270057Z NV_NVTX_VERSION=12.4.99-1 2024-08-08T20:49:43.4270855Z GITHUB_RUN_NUMBER=90547 2024-08-08T20:49:43.4271569Z TEST_CONFIG=default 2024-08-08T20:49:43.4271977Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-08T20:49:43.4272610Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-08T20:49:43.4273187Z NV_CUDA_CUDART_DEV_VERSION=12.4.99-1 2024-08-08T20:49:43.4273725Z NV_LIBCUSPARSE_VERSION=12.3.0.142-1 2024-08-08T20:49:43.4274260Z NV_LIBNPP_VERSION=12.2.5.2-1 2024-08-08T20:49:43.4274824Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-08T20:49:43.4275617Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-08T20:49:43.4276229Z GITHUB_REF_TYPE=tag 2024-08-08T20:49:43.4276630Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-08T20:49:43.4277098Z NCCL_VERSION=2.20.5-1 2024-08-08T20:49:43.4277578Z BASE_SHA=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:43.4278125Z XLA_CUDA= 2024-08-08T20:49:43.4278453Z HUGGING_FACE_HUB_TOKEN= 2024-08-08T20:49:43.4281033Z *** 2024-08-08T20:49:43.4281411Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-08T20:49:43.4281912Z GITHUB_REPOSITORY_ID=65600975 2024-08-08T20:49:43.4282365Z GITHUB_ACTIONS=true 2024-08-08T20:49:43.4282775Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:49:43.4283445Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.99-1 2024-08-08T20:49:43.4284126Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.2-1 2024-08-08T20:49:43.4284704Z SHA1=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:43.4285334Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-08T20:49:43.4285946Z GITHUB_SHA=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:43.4286953Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/trunk.yml@refs/tags/ciflow/trunk/132710 2024-08-08T20:49:43.4287795Z UCC_HOME=/usr 2024-08-08T20:49:43.4288227Z NV_LIBCUBLAS_DEV_VERSION=12.4.2.65-1 2024-08-08T20:49:43.4288731Z VERBOSE_TEST_LOGS=False 2024-08-08T20:49:43.4289141Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-08T20:49:43.4289731Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4 2024-08-08T20:49:43.4290347Z GITHUB_REF=refs/tags/ciflow/trunk/132710 2024-08-08T20:49:43.4290915Z NV_CUDA_CUDART_VERSION=12.4.99-1 2024-08-08T20:49:43.4291380Z SHARD_NUMBER=4 2024-08-08T20:49:43.4291761Z GITHUB_REF_PROTECTED=false 2024-08-08T20:49:43.4292173Z HOME=/var/lib/jenkins 2024-08-08T20:49:43.4292625Z GITHUB_API_URL=https://api.github.com 2024-08-08T20:49:43.4293182Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-08T20:49:43.4293754Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-08T20:49:43.4294353Z SCCACHE_S3_KEY_PREFIX=trunk 2024-08-08T20:49:43.4294805Z CUDA_VERSION=12.4.0 2024-08-08T20:49:43.4295334Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.2.65-1 2024-08-08T20:49:43.4295887Z NUM_TEST_SHARDS=5 2024-08-08T20:49:43.4296258Z UCX_HOME=/usr 2024-08-08T20:49:43.4296900Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.0-1 2024-08-08T20:49:43.4298293Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4300186Z JOB_NAME=linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build-test / test (default, 4, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-08T20:49:43.4302038Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4303496Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-08T20:49:43.4304329Z GITHUB_EVENT_NAME=push 2024-08-08T20:49:43.4304732Z DASHBOARD_TAG= 2024-08-08T20:49:43.4305084Z GITHUB_RUN_ID=10308807531 2024-08-08T20:49:43.4305673Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.2-1 2024-08-08T20:49:43.4306356Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4 2024-08-08T20:49:43.4307630Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4308808Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-08T20:49:43.4309323Z NV_LIBNPP_DEV_VERSION=12.2.5.2-1 2024-08-08T20:49:43.4309763Z PR_NUMBER= 2024-08-08T20:49:43.4310109Z GITHUB_RUN_ATTEMPT=1 2024-08-08T20:49:43.4310703Z ANACONDA_PYTHON_VERSION=3.10 2024-08-08T20:49:43.4311236Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-08T20:49:43.4311805Z TERM=xterm 2024-08-08T20:49:43.4312231Z NV_LIBCUSPARSE_DEV_VERSION=12.3.0.142-1 2024-08-08T20:49:43.4312730Z INSTALLED_VISION=yes 2024-08-08T20:49:43.4313108Z BRANCH= 2024-08-08T20:49:43.4313453Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-08T20:49:43.4313931Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-08T20:49:43.4314454Z CUDA_PATH=/usr/local/cuda 2024-08-08T20:49:43.4315547Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-08T20:49:43.4316506Z GITHUB_SERVER_URL=https://github.com 2024-08-08T20:49:43.4317112Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-08T20:49:43.4317699Z REENABLED_ISSUES= 2024-08-08T20:49:43.4318045Z SHLVL=1 2024-08-08T20:49:43.4318365Z MAX_JOBS=14 2024-08-08T20:49:43.4318765Z NV_CUDA_LIB_VERSION=12.4.0-1 2024-08-08T20:49:43.4319183Z NVARCH=x86_64 2024-08-08T20:49:43.4319558Z GITHUB_ACTOR_ID=54816060 2024-08-08T20:49:43.4320117Z GITHUB_WORKFLOW_SHA=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:43.4320753Z GITHUB_REF_NAME=ciflow/trunk/132710 2024-08-08T20:49:43.4321359Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4 2024-08-08T20:49:43.4322208Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-08T20:49:43.4322907Z GITHUB_JOB=test 2024-08-08T20:49:43.4323412Z NV_LIBNCCL_PACKAGE=libnccl2=2.20.5-1+cuda12.4 2024-08-08T20:49:43.4324116Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-08T20:49:43.4324747Z NO_TEST_TIMEOUT=False 2024-08-08T20:49:43.4325147Z TD_DISTRIBUTED=False 2024-08-08T20:49:43.4325626Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.0-1 2024-08-08T20:49:43.4326156Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-08T20:49:43.4326679Z NV_NVPROF_VERSION=12.4.99-1 2024-08-08T20:49:43.4327124Z GITHUB_RETENTION_DAYS=90 2024-08-08T20:49:43.4327534Z OPENSSL_DIR=/opt/openssl 2024-08-08T20:49:43.4327966Z GITHUB_ACTION_REPOSITORY= 2024-08-08T20:49:43.4329306Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-08T20:49:43.4330658Z GITHUB_BASE_REF= 2024-08-08T20:49:43.4331057Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-08T20:49:43.4331510Z CI=true 2024-08-08T20:49:43.4331889Z NV_LIBNCCL_PACKAGE_VERSION=2.20.5-1 2024-08-08T20:49:43.4332402Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-08T20:49:43.4332870Z JOB_ID=28538244287 2024-08-08T20:49:43.4333240Z INSTALLED_PROTOBUF=yes 2024-08-08T20:49:43.4333644Z GITHUB_HEAD_REF= 2024-08-08T20:49:43.4334014Z GITHUB_ACTION_REF= 2024-08-08T20:49:43.4334550Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-08T20:49:43.4335138Z TEST_SHOWLOCALS=False 2024-08-08T20:49:43.4335543Z GITHUB_WORKFLOW=trunk 2024-08-08T20:49:43.4335949Z DEBIAN_FRONTEND=noninteractive 2024-08-08T20:49:43.4337126Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4338175Z NO_TD=False 2024-08-08T20:49:43.4338529Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-08T20:49:43.4338974Z _=/usr/bin/env 2024-08-08T20:49:43.4339592Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-08T20:49:43.4479774Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-08T20:49:43.4481346Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-08T20:49:43.4482865Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-08T20:49:43.4484873Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-08T20:49:43.4485674Z + BUILD_DIR=build 2024-08-08T20:49:43.4486327Z + BUILD_RENAMED_DIR=build_renamed 2024-08-08T20:49:43.4486812Z + BUILD_BIN_DIR=build/bin 2024-08-08T20:49:43.4487291Z + SHARD_NUMBER=4 2024-08-08T20:49:43.4487577Z + NUM_TEST_SHARDS=5 2024-08-08T20:49:43.4488085Z + export VALGRIND=ON 2024-08-08T20:49:43.4488668Z + VALGRIND=ON 2024-08-08T20:49:43.4490018Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *clang9* ]] 2024-08-08T20:49:43.4491110Z + [[ 0 == \1 ]] 2024-08-08T20:49:43.4491458Z + [[ False == \1 ]] 2024-08-08T20:49:43.4492266Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build != *bazel* ]] 2024-08-08T20:49:43.4493196Z ++ realpath build/custom_test_artifacts 2024-08-08T20:49:43.4500462Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-08T20:49:43.4501530Z + [[ -n '' ]] 2024-08-08T20:49:43.4501883Z + echo 'Environment variables' 2024-08-08T20:49:43.4502335Z Environment variables 2024-08-08T20:49:43.4502811Z + env 2024-08-08T20:49:43.4508832Z INSTALLED_DB=yes 2024-08-08T20:49:43.4509520Z NV_LIBCUBLAS_VERSION=12.4.2.65-1 2024-08-08T20:49:43.4510429Z NVIDIA_VISIBLE_DEVICES=all 2024-08-08T20:49:43.4511574Z NV_NVML_DEV_VERSION=12.4.99-1 2024-08-08T20:49:43.4512589Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-08T20:49:43.4513309Z CONTINUE_THROUGH_ERROR=False 2024-08-08T20:49:43.4514058Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.20.5-1+cuda12.4 2024-08-08T20:49:43.4514770Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.20.5-1 2024-08-08T20:49:43.4515717Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build 2024-08-08T20:49:43.4516883Z HOSTNAME=eca2856b64b6 2024-08-08T20:49:43.4518556Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4519756Z GITHUB_ACTION=__self 2024-08-08T20:49:43.4520521Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-08T20:49:43.4530179Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536 2024-08-08T20:49:43.4537928Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.2.65-1 2024-08-08T20:49:43.4538678Z NV_NVTX_VERSION=12.4.99-1 2024-08-08T20:49:43.4538994Z GITHUB_RUN_NUMBER=90547 2024-08-08T20:49:43.4539414Z TEST_CONFIG=default 2024-08-08T20:49:43.4539924Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-08T20:49:43.4540407Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-08T20:49:43.4541076Z NV_CUDA_CUDART_DEV_VERSION=12.4.99-1 2024-08-08T20:49:43.4541531Z NV_LIBCUSPARSE_VERSION=12.3.0.142-1 2024-08-08T20:49:43.4542039Z NV_LIBNPP_VERSION=12.2.5.2-1 2024-08-08T20:49:43.4542597Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-08T20:49:43.4543210Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-08T20:49:43.4543877Z GITHUB_REF_TYPE=tag 2024-08-08T20:49:43.4544180Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-08T20:49:43.4544656Z NCCL_VERSION=2.20.5-1 2024-08-08T20:49:43.4545114Z BASE_SHA=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:43.4545664Z XLA_CUDA= 2024-08-08T20:49:43.4546111Z HUGGING_FACE_HUB_TOKEN= 2024-08-08T20:49:43.4546493Z *** 2024-08-08T20:49:43.4546838Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-08T20:49:43.4547328Z GITHUB_REPOSITORY_ID=65600975 2024-08-08T20:49:43.4547759Z GITHUB_ACTIONS=true 2024-08-08T20:49:43.4548329Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T20:49:43.4548957Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.99-1 2024-08-08T20:49:43.4549607Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.2-1 2024-08-08T20:49:43.4550182Z SHA1=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:43.4550825Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-08T20:49:43.4551408Z GITHUB_SHA=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:43.4552445Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/trunk.yml@refs/tags/ciflow/trunk/132710 2024-08-08T20:49:43.4553323Z UCC_HOME=/usr 2024-08-08T20:49:43.4553729Z NV_LIBCUBLAS_DEV_VERSION=12.4.2.65-1 2024-08-08T20:49:43.4554232Z VERBOSE_TEST_LOGS=False 2024-08-08T20:49:43.4554649Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-08T20:49:43.4555209Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4 2024-08-08T20:49:43.4555824Z GITHUB_REF=refs/tags/ciflow/trunk/132710 2024-08-08T20:49:43.4556388Z NV_CUDA_CUDART_VERSION=12.4.99-1 2024-08-08T20:49:43.4556830Z SHARD_NUMBER=4 2024-08-08T20:49:43.4557218Z GITHUB_REF_PROTECTED=false 2024-08-08T20:49:43.4557647Z HOME=/var/lib/jenkins 2024-08-08T20:49:43.4558076Z GITHUB_API_URL=https://api.github.com 2024-08-08T20:49:43.4559056Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-08T20:49:43.4559658Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-08T20:49:43.4560234Z SCCACHE_S3_KEY_PREFIX=trunk 2024-08-08T20:49:43.4560722Z CUDA_VERSION=12.4.0 2024-08-08T20:49:43.4561281Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.2.65-1 2024-08-08T20:49:43.4561842Z NUM_TEST_SHARDS=5 2024-08-08T20:49:43.4562193Z UCX_HOME=/usr 2024-08-08T20:49:43.4562848Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.0-1 2024-08-08T20:49:43.4564241Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4566127Z JOB_NAME=linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build-test / test (default, 4, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-08T20:49:43.4567977Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4569412Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-08T20:49:43.4570236Z GITHUB_EVENT_NAME=push 2024-08-08T20:49:43.4570663Z DASHBOARD_TAG= 2024-08-08T20:49:43.4570987Z GITHUB_RUN_ID=10308807531 2024-08-08T20:49:43.4571568Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.2-1 2024-08-08T20:49:43.4572235Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4 2024-08-08T20:49:43.4573525Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4574693Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-08T20:49:43.4575185Z NV_LIBNPP_DEV_VERSION=12.2.5.2-1 2024-08-08T20:49:43.4575641Z PR_NUMBER= 2024-08-08T20:49:43.4575985Z GITHUB_RUN_ATTEMPT=1 2024-08-08T20:49:43.4576354Z VALGRIND=ON 2024-08-08T20:49:43.4576718Z ANACONDA_PYTHON_VERSION=3.10 2024-08-08T20:49:43.4577283Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-08T20:49:43.4577841Z TERM=xterm 2024-08-08T20:49:43.4578260Z NV_LIBCUSPARSE_DEV_VERSION=12.3.0.142-1 2024-08-08T20:49:43.4578772Z INSTALLED_VISION=yes 2024-08-08T20:49:43.4579133Z BRANCH= 2024-08-08T20:49:43.4579472Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-08T20:49:43.4579962Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-08T20:49:43.4580468Z CUDA_PATH=/usr/local/cuda 2024-08-08T20:49:43.4581445Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-08T20:49:43.4582404Z GITHUB_SERVER_URL=https://github.com 2024-08-08T20:49:43.4582991Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-08T20:49:43.4583571Z REENABLED_ISSUES= 2024-08-08T20:49:43.4583937Z SHLVL=1 2024-08-08T20:49:43.4584233Z MAX_JOBS=14 2024-08-08T20:49:43.4584627Z NV_CUDA_LIB_VERSION=12.4.0-1 2024-08-08T20:49:43.4585060Z NVARCH=x86_64 2024-08-08T20:49:43.4585404Z GITHUB_ACTOR_ID=54816060 2024-08-08T20:49:43.4586150Z GITHUB_WORKFLOW_SHA=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-08T20:49:43.4586853Z GITHUB_REF_NAME=ciflow/trunk/132710 2024-08-08T20:49:43.4587446Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4 2024-08-08T20:49:43.4588293Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-08T20:49:43.4589014Z GITHUB_JOB=test 2024-08-08T20:49:43.4589499Z NV_LIBNCCL_PACKAGE=libnccl2=2.20.5-1+cuda12.4 2024-08-08T20:49:43.4590301Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-08T20:49:43.4590961Z NO_TEST_TIMEOUT=False 2024-08-08T20:49:43.4591344Z TD_DISTRIBUTED=False 2024-08-08T20:49:43.4591822Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.0-1 2024-08-08T20:49:43.4592364Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-08T20:49:43.4592871Z NV_NVPROF_VERSION=12.4.99-1 2024-08-08T20:49:43.4593319Z GITHUB_RETENTION_DAYS=90 2024-08-08T20:49:43.4593748Z OPENSSL_DIR=/opt/openssl 2024-08-08T20:49:43.4594155Z GITHUB_ACTION_REPOSITORY= 2024-08-08T20:49:43.4595476Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-08T20:49:43.4596948Z GITHUB_BASE_REF= 2024-08-08T20:49:43.4597254Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-08T20:49:43.4597711Z CI=true 2024-08-08T20:49:43.4598107Z NV_LIBNCCL_PACKAGE_VERSION=2.20.5-1 2024-08-08T20:49:43.4598601Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-08T20:49:43.4599061Z JOB_ID=28538244287 2024-08-08T20:49:43.4599451Z INSTALLED_PROTOBUF=yes 2024-08-08T20:49:43.4599838Z GITHUB_HEAD_REF= 2024-08-08T20:49:43.4600206Z GITHUB_ACTION_REF= 2024-08-08T20:49:43.4600764Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-08T20:49:43.4601329Z TEST_SHOWLOCALS=False 2024-08-08T20:49:43.4601730Z GITHUB_WORKFLOW=trunk 2024-08-08T20:49:43.4602149Z DEBIAN_FRONTEND=noninteractive 2024-08-08T20:49:43.4603309Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_ee555a89-7f48-4d72-ac08-cf95968782dd 2024-08-08T20:49:43.4604355Z NO_TD=False 2024-08-08T20:49:43.4604721Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-08T20:49:43.4605151Z _=/usr/bin/env 2024-08-08T20:49:43.4605581Z + echo 'Testing pytorch' 2024-08-08T20:49:43.4605998Z Testing pytorch 2024-08-08T20:49:43.4606382Z + export LANG=C.UTF-8 2024-08-08T20:49:43.4606799Z + LANG=C.UTF-8 2024-08-08T20:49:43.4607154Z + PR_NUMBER= 2024-08-08T20:49:43.4607508Z + [[ default == \d\e\f\a\u\l\t ]] 2024-08-08T20:49:43.4607995Z + export CUDA_VISIBLE_DEVICES=0 2024-08-08T20:49:43.4608471Z + CUDA_VISIBLE_DEVICES=0 2024-08-08T20:49:43.4608893Z + export HIP_VISIBLE_DEVICES=0 2024-08-08T20:49:43.4609356Z + HIP_VISIBLE_DEVICES=0 2024-08-08T20:49:43.4609802Z + [[ default == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-08T20:49:43.4610307Z + [[ default == \s\l\o\w ]] 2024-08-08T20:49:43.4611225Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *slow-gradcheck* ]] 2024-08-08T20:49:43.4612493Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *cuda* ]] 2024-08-08T20:49:43.4613350Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-08T20:49:43.4613939Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-08T20:49:43.4614468Z + [[ default == *crossref* ]] 2024-08-08T20:49:43.4615305Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *rocm* ]] 2024-08-08T20:49:43.4616481Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *xpu* ]] 2024-08-08T20:49:43.4617674Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build != *-bazel-* ]] 2024-08-08T20:49:43.4618558Z + pip_install --user ninja==1.10.2 2024-08-08T20:49:43.4619242Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-08T20:49:44.1090057Z Collecting ninja==1.10.2 2024-08-08T20:49:44.1306071Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-08T20:49:44.1494751Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-08T20:49:45.3851385Z Installing collected packages: ninja 2024-08-08T20:49:45.3960773Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-08T20:49:45.3962541Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-08T20:49:45.4025712Z Successfully installed ninja-1.10.2 2024-08-08T20:49:45.4737805Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-08T20:49:45.4740756Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-08T20:49:45.4742797Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *aarch64* ]] 2024-08-08T20:49:45.4743604Z + install_tlparse 2024-08-08T20:49:45.4744199Z + pip_install --user tlparse==0.3.7 2024-08-08T20:49:45.4744824Z + pip install --progress-bar off --user tlparse==0.3.7 2024-08-08T20:49:46.0067815Z Collecting tlparse==0.3.7 2024-08-08T20:49:46.0249581Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-08-08T20:49:46.0348173Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-08T20:49:47.2891683Z Installing collected packages: tlparse 2024-08-08T20:49:47.3384352Z Successfully installed tlparse-0.3.7 2024-08-08T20:49:47.4103088Z ++ python -m site --user-base 2024-08-08T20:49:47.4339974Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-08T20:49:47.4342805Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *asan* ]] 2024-08-08T20:49:47.4344651Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *-debug* ]] 2024-08-08T20:49:47.4345898Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build != *-bazel-* ]] 2024-08-08T20:49:47.4348435Z + echo 'We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build. Expect the assertion to pass' 2024-08-08T20:49:47.4350425Z We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build. Expect the assertion to pass 2024-08-08T20:49:47.4351500Z + cd test 2024-08-08T20:49:47.4352175Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-08T20:49:49.5856665Z + [[ default == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-08T20:49:49.5857357Z + [[ default == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-08T20:49:49.5863268Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-08T20:49:49.5864363Z + [[ default == *dynamo_eager* ]] 2024-08-08T20:49:49.5864986Z + [[ default == *aot_eager* ]] 2024-08-08T20:49:49.5865515Z + [[ default == *aot_inductor* ]] 2024-08-08T20:49:49.5865943Z + [[ default == *inductor* ]] 2024-08-08T20:49:49.5866449Z + [[ default == *dynamic* ]] 2024-08-08T20:49:49.5866882Z + [[ default == *cpu* ]] 2024-08-08T20:49:49.5867634Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-08T20:49:49.5905767Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *libtorch* ]] 2024-08-08T20:49:49.5906943Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *-bazel-* ]] 2024-08-08T20:49:49.5908649Z + cd test 2024-08-08T20:49:49.5909287Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-08T20:49:51.5056043Z PyTorch built with: 2024-08-08T20:49:51.5057165Z - GCC 9.4 2024-08-08T20:49:51.5057588Z - C++ Version: 201703 2024-08-08T20:49:51.5058947Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-08T20:49:51.5060347Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-08T20:49:51.5061208Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-08T20:49:51.5061866Z - LAPACK is enabled (usually provided by MKL) 2024-08-08T20:49:51.5062897Z - NNPACK is enabled 2024-08-08T20:49:51.5063367Z - CPU capability usage: AVX2 2024-08-08T20:49:51.5063885Z - CUDA Runtime 12.4 2024-08-08T20:49:51.5064551Z - NVCC architecture flags: -gencode;arch=compute_52,code=sm_52 2024-08-08T20:49:51.5065235Z - CuDNN 90.1 2024-08-08T20:49:51.5065636Z - Magma 2.6.1 2024-08-08T20:49:51.5074846Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.4, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-08T20:49:51.5083193Z 2024-08-08T20:49:51.7770406Z + cd test 2024-08-08T20:49:51.7771323Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-08T20:49:53.6769858Z ATen/Parallel: 2024-08-08T20:49:53.6770874Z at::get_num_threads() : 8 2024-08-08T20:49:53.6771589Z at::get_num_interop_threads() : 8 2024-08-08T20:49:53.6771986Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-08T20:49:53.6772509Z omp_get_max_threads() : 8 2024-08-08T20:49:53.6773792Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-08T20:49:53.6774798Z mkl_get_max_threads() : 8 2024-08-08T20:49:53.6775592Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-08T20:49:53.6776386Z std::thread::hardware_concurrency() : 16 2024-08-08T20:49:53.6776906Z Environment variables: 2024-08-08T20:49:53.6777353Z OMP_NUM_THREADS : [not set] 2024-08-08T20:49:53.6777817Z MKL_NUM_THREADS : [not set] 2024-08-08T20:49:53.6778265Z ATen parallel backend: OpenMP 2024-08-08T20:49:53.6778635Z 2024-08-08T20:49:53.9357786Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *aarch64* ]] 2024-08-08T20:49:53.9359215Z + [[ default == *backward* ]] 2024-08-08T20:49:53.9359671Z + [[ default == *xla* ]] 2024-08-08T20:49:53.9360021Z + [[ default == *executorch* ]] 2024-08-08T20:49:53.9360503Z + [[ default == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-08T20:49:53.9361512Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *libtorch* ]] 2024-08-08T20:49:53.9362784Z + [[ default == distributed ]] 2024-08-08T20:49:53.9363326Z + [[ default == *inductor_distributed* ]] 2024-08-08T20:49:53.9363923Z + [[ default == *inductor-halide* ]] 2024-08-08T20:49:53.9364598Z + [[ default == *inductor-micro-benchmark* ]] 2024-08-08T20:49:53.9365181Z + [[ default == *huggingface* ]] 2024-08-08T20:49:53.9365636Z + [[ default == *timm* ]] 2024-08-08T20:49:53.9365997Z + [[ default == *torchbench* ]] 2024-08-08T20:49:53.9366591Z + [[ default == *inductor_cpp_wrapper_abi_compatible* ]] 2024-08-08T20:49:53.9367211Z + [[ default == *inductor* ]] 2024-08-08T20:49:53.9367653Z + [[ default == *dynamo* ]] 2024-08-08T20:49:53.9368500Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build == *rocm* ]] 2024-08-08T20:49:53.9369284Z + [[ 4 == 1 ]] 2024-08-08T20:49:53.9369616Z + [[ 4 == 2 ]] 2024-08-08T20:49:53.9370017Z + [[ 4 -gt 2 ]] 2024-08-08T20:49:53.9370772Z + install_torchvision 2024-08-08T20:49:53.9371155Z + local orig_preload 2024-08-08T20:49:53.9371542Z + local commit 2024-08-08T20:49:53.9371930Z ++ get_pinned_commit vision 2024-08-08T20:49:53.9372396Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-08T20:49:53.9382049Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-08T20:49:53.9382620Z + orig_preload= 2024-08-08T20:49:53.9383074Z + '[' -n '' ']' 2024-08-08T20:49:53.9384297Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-08T20:49:53.9386265Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-08T20:49:54.4144477Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-08T20:49:54.4152222Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-w8l3skoz 2024-08-08T20:49:54.4179181Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-w8l3skoz 2024-08-08T20:49:56.0817164Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-08T20:49:56.0843774Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-08T20:49:57.7596875Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-08T20:49:58.0212053Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-08T20:50:01.0006439Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-08T20:50:01.0069781Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-08-08T20:50:01.0075144Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb9d86fa) 2024-08-08T20:50:01.0083625Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-08T20:50:01.0419810Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-08T20:50:01.0430693Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-08T20:50:01.0435673Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-08T20:50:01.0440567Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-08T20:50:01.0445745Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-08T20:50:01.0455026Z Requirement already satisfied: torch-no-python==2.5.0a0+gitb9d86fa in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb9d86fa) 2024-08-08T20:50:01.0469321Z Requirement already satisfied: sympy>=1.13.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-08T20:50:01.0594836Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.0->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-08T20:50:01.1891052Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-08T20:50:01.2213302Z Building wheels for collected packages: torchvision 2024-08-08T20:51:45.7770380Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-08-08T20:51:45.7846404Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2024564 sha256=e16f88ff665a048c51186d167f33052ba110bbb5b9d9b6949a55a795eab18f02 2024-08-08T20:51:45.7848933Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-08T20:51:45.7900643Z Successfully built torchvision 2024-08-08T20:51:46.8266135Z Installing collected packages: torchvision 2024-08-08T20:51:47.4135637Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-08T20:51:47.5395289Z + '[' -n '' ']' 2024-08-08T20:51:47.5395970Z + test_python_shard 4 2024-08-08T20:51:47.5399371Z + [[ -z 5 ]] 2024-08-08T20:51:47.5400372Z + python test/run_test.py --exclude-jit-executor --exclude-distributed-tests --shard 4 5 --verbose 2024-08-08T20:51:47.6735334Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-08T20:51:47.6736794Z import pkg_resources 2024-08-08T20:51:52.2965070Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-08T20:51:52.3628719Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-08T20:51:52.4085468Z Ignoring disabled issues: [''] 2024-08-08T20:51:52.4276411Z ##[warning] Gathered no stats from artifacts for linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build build env and default test config. Using default build env and default test config instead. 2024-08-08T20:51:52.4799378Z ##[warning] Gathered no stats from artifacts for linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build build env and default test config. Using default build env and default test config instead. 2024-08-08T20:51:52.4812920Z Running 25% of tests based on TD 2024-08-08T20:51:52.5208257Z Running parallel tests on 2 processes 2024-08-08T20:51:52.5211149Z Name: tests to run (est. time: 82.02min) 2024-08-08T20:51:52.5211706Z Serial tests (0): 2024-08-08T20:51:52.5212119Z Parallel tests (27): 2024-08-08T20:51:52.5212537Z test_decomp 1/7 2024-08-08T20:51:52.5212962Z test_model_exports_to_core_aten 1/1 2024-08-08T20:51:52.5213495Z test_transformers 1/1 2024-08-08T20:51:52.5213997Z inductor/test_torchinductor_opinfo 1/18 2024-08-08T20:51:52.5214571Z inductor/test_torchinductor_opinfo 6/18 2024-08-08T20:51:52.5215166Z inductor/test_torchinductor_opinfo 11/18 2024-08-08T20:51:52.5219030Z inductor/test_torchinductor_opinfo 16/18 2024-08-08T20:51:52.5220294Z functorch/test_ops 4/5 2024-08-08T20:51:52.5220832Z dynamo/test_dynamic_shapes 1/2 2024-08-08T20:51:52.5221336Z dynamo/test_dynamic_shapes 2/2 2024-08-08T20:51:52.5221750Z test_ops 4/4 2024-08-08T20:51:52.5222180Z test_ops_jit 2/2 2024-08-08T20:51:52.5222567Z inductor/test_aot_inductor 4/9 2024-08-08T20:51:52.5223531Z test_mkldnn_verbose 1/1 2024-08-08T20:51:52.5224521Z test_expanded_weights 1/1 2024-08-08T20:51:52.5225019Z test_ops_fwd_gradients 1/2 2024-08-08T20:51:52.5225477Z test_ops_fwd_gradients 2/2 2024-08-08T20:51:52.5225974Z inductor/test_cuda_cpp_wrapper 2/3 2024-08-08T20:51:52.5226557Z inductor/test_indexing 1/1 2024-08-08T20:51:52.5226958Z dynamo/test_nops 1/1 2024-08-08T20:51:52.5227381Z inductor/test_triton_heuristics 1/1 2024-08-08T20:51:52.5227895Z inductor/test_foreach 1/1 2024-08-08T20:51:52.5228376Z inductor/test_inductor_freezing 1/1 2024-08-08T20:51:52.5228953Z inductor/test_fx_fusion 1/1 2024-08-08T20:51:52.5229379Z inductor/test_cpu_cpp_wrapper 1/13 2024-08-08T20:51:52.5230014Z inductor/test_cpu_cpp_wrapper 10/13 2024-08-08T20:51:52.5230817Z inductor/test_cpu_cpp_wrapper 11/13 2024-08-08T20:51:52.5231347Z Name: excluded (est. time: 43.55min) 2024-08-08T20:51:52.5231832Z Serial tests (0): 2024-08-08T20:51:52.5232226Z Parallel tests (52): 2024-08-08T20:51:52.5232630Z test_quantization 3/4 2024-08-08T20:51:52.5233071Z inductor/test_halide 5/5 2024-08-08T20:51:52.5233615Z inductor/test_cpu_select_algorithm 1/11 2024-08-08T20:51:52.5234129Z inductor/test_cpu_select_algorithm 5/11 2024-08-08T20:51:52.5234939Z inductor/test_cpu_select_algorithm 6/11 2024-08-08T20:51:52.5235458Z inductor/test_torchinductor 2/3 2024-08-08T20:51:52.5236049Z inductor/test_torchinductor 3/3 2024-08-08T20:51:52.5236453Z test_sparse_csr 1/1 2024-08-08T20:51:52.5236870Z test_dataloader 1/1 2024-08-08T20:51:52.5237323Z test_binary_ufuncs 1/1 2024-08-08T20:51:52.5237735Z inductor/test_split_cat_fx_passes 1/1 2024-08-08T20:51:52.5238269Z test_fx_experimental 1/1 2024-08-08T20:51:52.5238726Z inductor/test_cuda_repro 1/1 2024-08-08T20:51:52.5239226Z inductor/test_minifier_isolate 1/1 2024-08-08T20:51:52.5239743Z test_serialization 1/1 2024-08-08T20:51:52.5240161Z dynamo/test_repros 1/1 2024-08-08T20:51:52.5240601Z profiler/test_profiler 1/1 2024-08-08T20:51:52.5241081Z dynamo/test_higher_order_ops 1/1 2024-08-08T20:51:52.5241596Z inductor/test_standalone_compile 1/1 2024-08-08T20:51:52.5242120Z export/test_passes 1/1 2024-08-08T20:51:52.5242619Z torch_np/numpy_tests/core/test_einsum 1/1 2024-08-08T20:51:52.5243244Z test_maskedtensor 1/1 2024-08-08T20:51:52.5243644Z inductor/test_custom_post_grad_passes 1/1 2024-08-08T20:51:52.5244203Z inductor/test_layout_optim 1/1 2024-08-08T20:51:52.5244682Z dynamo/test_decorators 1/1 2024-08-08T20:51:52.5245137Z test_stateless 1/1 2024-08-08T20:51:52.5245640Z test_dynamic_shapes 1/1 2024-08-08T20:51:52.5245973Z test_content_store 1/1 2024-08-08T20:51:52.5246410Z test_public_bindings 1/1 2024-08-08T20:51:52.5246892Z test_weak 1/1 2024-08-08T20:51:52.5247231Z export/test_unflatten 1/1 2024-08-08T20:51:52.5247704Z torch_np/test_ufuncs_basic 1/1 2024-08-08T20:51:52.5248207Z backends/xeon/test_launch 1/1 2024-08-08T20:51:52.5248665Z export/test_db 1/1 2024-08-08T20:51:52.5249151Z test_dlpack 1/1 2024-08-08T20:51:52.5249463Z nn/test_packed_sequence 1/1 2024-08-08T20:51:52.5249926Z test_tensorexpr_pybind 1/1 2024-08-08T20:51:52.5250475Z test_functionalization_of_rng_ops 1/1 2024-08-08T20:51:52.5250982Z dynamo/test_recompile_ux 1/1 2024-08-08T20:51:52.5251540Z export/test_pass_infra 1/1 2024-08-08T20:51:52.5251915Z dynamo/test_optimizers 1/1 2024-08-08T20:51:52.5252375Z test_flop_counter 1/1 2024-08-08T20:51:52.5252875Z dynamo/test_config 1/1 2024-08-08T20:51:52.5253241Z dynamo/test_export_mutations 1/1 2024-08-08T20:51:52.5253748Z test_out_dtype_op 1/1 2024-08-08T20:51:52.5254159Z dynamo/test_sources 1/1 2024-08-08T20:51:52.5254621Z functorch/test_logging 1/1 2024-08-08T20:51:52.5255182Z torch_np/test_function_base 1/1 2024-08-08T20:51:52.5255586Z dynamo/test_guard_manager 1/1 2024-08-08T20:51:52.5256073Z dynamo/test_debug_utils 1/1 2024-08-08T20:51:52.5256532Z xpu/test_gemm 1/1 2024-08-08T20:51:52.5256912Z optim/test_optim 1/1 2024-08-08T20:51:52.5257737Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-08T20:51:52.5295587Z Running test_decomp 1/7 ... [2024-08-08 20:51:52.529064] 2024-08-08T20:51:52.5301712Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_decomp.py', '-m', 'serial', '--shard-id=1', '--num-shards=7', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:51:52.529527] 2024-08-08T20:52:00.4100215Z 2024-08-08T20:52:00.4101831Z test_decomp 1/7 was successful, full logs can be found in artifacts with path test/test-reports/test_decomp_1.7_9f0c7d6a7abe2ca9_.log 2024-08-08T20:52:00.4104675Z Running 0 items in this shard: 2024-08-08T20:52:00.4104945Z 2024-08-08T20:52:00.4107361Z Running test_model_exports_to_core_aten 1/1 ... [2024-08-08 20:52:00.410378] 2024-08-08T20:52:00.4112926Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_model_exports_to_core_aten.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:52:00.410816] 2024-08-08T20:52:04.3830209Z 2024-08-08T20:52:04.3832271Z test_model_exports_to_core_aten 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_model_exports_to_core_aten_1.1_b61af7ea6bd55146_.log 2024-08-08T20:52:04.3834015Z Running 0 items in this shard: 2024-08-08T20:52:04.3834392Z 2024-08-08T20:52:04.3835382Z Running test_transformers 1/1 ... [2024-08-08 20:52:04.383232] 2024-08-08T20:52:04.3841584Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_transformers.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:52:04.383705] 2024-08-08T20:52:22.1800423Z 2024-08-08T20:52:22.1802323Z test_transformers 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_transformers_1.1_121b2bca58ee998a_.log 2024-08-08T20:52:22.1803885Z Running 0 items in this shard: 2024-08-08T20:52:22.1804241Z 2024-08-08T20:52:22.1805061Z Running inductor/test_torchinductor_opinfo 1/18 ... [2024-08-08 20:52:22.180195] 2024-08-08T20:52:22.1810962Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor_opinfo.py', '-m', 'serial', '--shard-id=1', '--num-shards=18', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:52:22.180642] 2024-08-08T20:52:34.3187636Z 2024-08-08T20:52:34.3189890Z inductor/test_torchinductor_opinfo 1/18 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_opinfo_1.18_44931cefcbd141ce_.log 2024-08-08T20:52:34.3191484Z Running 0 items in this shard: 2024-08-08T20:52:34.3192023Z 2024-08-08T20:52:34.3192635Z Running inductor/test_torchinductor_opinfo 6/18 ... [2024-08-08 20:52:34.318464] 2024-08-08T20:52:34.3195114Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor_opinfo.py', '-m', 'serial', '--shard-id=6', '--num-shards=18', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:52:34.318947] 2024-08-08T20:52:44.2527518Z 2024-08-08T20:52:44.2529707Z inductor/test_torchinductor_opinfo 6/18 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_opinfo_6.18_7338181b005d0439_.log 2024-08-08T20:52:44.2531432Z Running 0 items in this shard: 2024-08-08T20:52:44.2531809Z 2024-08-08T20:52:44.2533555Z Running inductor/test_torchinductor_opinfo 11/18 ... [2024-08-08 20:52:44.253010] 2024-08-08T20:52:44.2540403Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor_opinfo.py', '-m', 'serial', '--shard-id=11', '--num-shards=18', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:52:44.253528] 2024-08-08T20:52:54.0876068Z 2024-08-08T20:52:54.0878262Z inductor/test_torchinductor_opinfo 11/18 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_opinfo_11.18_e6b8445cf0ea2a95_.log 2024-08-08T20:52:54.0879773Z Running 0 items in this shard: 2024-08-08T20:52:54.0880108Z 2024-08-08T20:52:54.0882419Z Running inductor/test_torchinductor_opinfo 16/18 ... [2024-08-08 20:52:54.087858] 2024-08-08T20:52:54.0888782Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor_opinfo.py', '-m', 'serial', '--shard-id=16', '--num-shards=18', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:52:54.088344] 2024-08-08T20:53:03.9724516Z 2024-08-08T20:53:03.9726715Z inductor/test_torchinductor_opinfo 16/18 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_opinfo_16.18_d951f924edba608b_.log 2024-08-08T20:53:03.9728649Z Running 0 items in this shard: 2024-08-08T20:53:03.9729108Z 2024-08-08T20:53:03.9730338Z Running functorch/test_ops 4/5 ... [2024-08-08 20:53:03.972719] 2024-08-08T20:53:03.9736845Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'functorch/test_ops.py', '-m', 'serial', '--shard-id=4', '--num-shards=5', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:53:03.973213] 2024-08-08T20:53:12.1034663Z 2024-08-08T20:53:12.1036536Z functorch/test_ops 4/5 was successful, full logs can be found in artifacts with path test/test-reports/functorch.test_ops_4.5_b1198d87da135905_.log 2024-08-08T20:53:12.1037814Z Running 0 items in this shard: 2024-08-08T20:53:12.1038175Z 2024-08-08T20:53:12.1041388Z Running dynamo/test_dynamic_shapes 1/2 ... [2024-08-08 20:53:12.103760] 2024-08-08T20:53:12.1047708Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'dynamo/test_dynamic_shapes.py', '-m', 'serial', '--shard-id=1', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:53:12.104259] 2024-08-08T20:53:18.6812256Z 2024-08-08T20:53:18.6814277Z dynamo/test_dynamic_shapes 1/2 was successful, full logs can be found in artifacts with path test/test-reports/dynamo.test_dynamic_shapes_1.2_d4b209d5fa610097_.log 2024-08-08T20:53:18.6815665Z Running 0 items in this shard: 2024-08-08T20:53:18.6816253Z 2024-08-08T20:53:18.6818951Z Running dynamo/test_dynamic_shapes 2/2 ... [2024-08-08 20:53:18.681487] 2024-08-08T20:53:18.6824829Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'dynamo/test_dynamic_shapes.py', '-m', 'serial', '--shard-id=2', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:53:18.681983] 2024-08-08T20:53:24.9589011Z 2024-08-08T20:53:24.9591517Z dynamo/test_dynamic_shapes 2/2 was successful, full logs can be found in artifacts with path test/test-reports/dynamo.test_dynamic_shapes_2.2_59d31e2422d6ee5f_.log 2024-08-08T20:53:24.9592991Z Running 0 items in this shard: 2024-08-08T20:53:24.9593277Z 2024-08-08T20:53:24.9596269Z Running test_ops 4/4 ... [2024-08-08 20:53:24.959225] 2024-08-08T20:53:24.9603132Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_ops.py', '-m', 'serial', '--shard-id=4', '--num-shards=4', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:53:24.959768] 2024-08-08T20:53:39.4518717Z 2024-08-08T20:53:39.4520177Z test_ops 4/4 was successful, full logs can be found in artifacts with path test/test-reports/test_ops_4.4_7b3eb80f9d6251e7_.log 2024-08-08T20:53:39.4521345Z Running 0 items in this shard: 2024-08-08T20:53:39.4521669Z 2024-08-08T20:53:39.4524660Z Running test_ops_jit 2/2 ... [2024-08-08 20:53:39.452108] 2024-08-08T20:53:39.4530152Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_ops_jit.py', '-m', 'serial', '--shard-id=2', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:53:39.452576] 2024-08-08T20:53:44.9279451Z 2024-08-08T20:53:44.9281292Z test_ops_jit 2/2 was successful, full logs can be found in artifacts with path test/test-reports/test_ops_jit_2.2_ac3b1d7ee1e84871_.log 2024-08-08T20:53:44.9282533Z Running 0 items in this shard: 2024-08-08T20:53:44.9283589Z 2024-08-08T20:53:44.9284808Z Running inductor/test_aot_inductor 4/9 ... [2024-08-08 20:53:44.928158] 2024-08-08T20:53:44.9290891Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_aot_inductor.py', '-m', 'serial', '--shard-id=4', '--num-shards=9', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:53:44.928633] 2024-08-08T20:53:53.4098028Z 2024-08-08T20:53:53.4100167Z inductor/test_aot_inductor 4/9 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_aot_inductor_4.9_3a9348364ab7ccbd_.log 2024-08-08T20:53:53.4101440Z 2024-08-08T20:53:53.4103733Z Running test_mkldnn_verbose 1/1 ... [2024-08-08 20:53:53.410058] 2024-08-08T20:53:53.4109533Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_mkldnn_verbose.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:53:53.410545] 2024-08-08T20:53:56.9820511Z 2024-08-08T20:53:56.9822185Z test_mkldnn_verbose 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_mkldnn_verbose_1.1_65577b130c1a3f65_.log 2024-08-08T20:53:56.9823469Z Running 0 items in this shard: 2024-08-08T20:53:56.9823818Z 2024-08-08T20:53:56.9827593Z Running test_expanded_weights 1/1 ... [2024-08-08 20:53:56.982420] 2024-08-08T20:53:56.9833720Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_expanded_weights.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:53:56.982958] 2024-08-08T20:54:02.4583910Z 2024-08-08T20:54:02.4586326Z test_expanded_weights 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_expanded_weights_1.1_168622ef72da6c7d_.log 2024-08-08T20:54:02.4588494Z Running 0 items in this shard: 2024-08-08T20:54:02.4588790Z 2024-08-08T20:54:02.4590261Z Running test_ops_fwd_gradients 1/2 ... [2024-08-08 20:54:02.458679] 2024-08-08T20:54:02.4596216Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_ops_fwd_gradients.py', '-m', 'serial', '--shard-id=1', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:02.459168] 2024-08-08T20:54:08.5350791Z 2024-08-08T20:54:08.5352615Z test_ops_fwd_gradients 1/2 was successful, full logs can be found in artifacts with path test/test-reports/test_ops_fwd_gradients_1.2_3ce36d6129ae49d5_.log 2024-08-08T20:54:08.5353938Z Running 0 items in this shard: 2024-08-08T20:54:08.5354315Z 2024-08-08T20:54:08.5355633Z Running test_ops_fwd_gradients 2/2 ... [2024-08-08 20:54:08.535271] 2024-08-08T20:54:08.5362562Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_ops_fwd_gradients.py', '-m', 'serial', '--shard-id=2', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:08.535712] 2024-08-08T20:54:14.6115129Z 2024-08-08T20:54:14.6117227Z test_ops_fwd_gradients 2/2 was successful, full logs can be found in artifacts with path test/test-reports/test_ops_fwd_gradients_2.2_7fdf1936b4363764_.log 2024-08-08T20:54:14.6118583Z Running 0 items in this shard: 2024-08-08T20:54:14.6118991Z 2024-08-08T20:54:14.6120158Z Running inductor/test_cuda_cpp_wrapper 2/3 ... [2024-08-08 20:54:14.611722] 2024-08-08T20:54:14.6126136Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_cuda_cpp_wrapper.py', '-m', 'serial', '--shard-id=2', '--num-shards=3', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:14.612155] 2024-08-08T20:54:22.3918678Z 2024-08-08T20:54:22.3920684Z inductor/test_cuda_cpp_wrapper 2/3 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_cuda_cpp_wrapper_2.3_06f8e2cded401208_.log 2024-08-08T20:54:22.3922351Z 2024-08-08T20:54:22.3923212Z Running inductor/test_indexing 1/1 ... [2024-08-08 20:54:22.392026] 2024-08-08T20:54:22.3929608Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_indexing.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:22.392486] 2024-08-08T20:54:29.1203602Z 2024-08-08T20:54:29.1205585Z inductor/test_indexing 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_indexing_1.1_ca45254af25dbed8_.log 2024-08-08T20:54:29.1207208Z Running 0 items in this shard: 2024-08-08T20:54:29.1207618Z 2024-08-08T20:54:29.1208920Z Running dynamo/test_nops 1/1 ... [2024-08-08 20:54:29.120564] 2024-08-08T20:54:29.1215083Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'dynamo/test_nops.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:29.121042] 2024-08-08T20:54:32.8930445Z 2024-08-08T20:54:32.8932365Z dynamo/test_nops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/dynamo.test_nops_1.1_380789cafc33d817_.log 2024-08-08T20:54:32.8933952Z Running 0 items in this shard: 2024-08-08T20:54:32.8934327Z 2024-08-08T20:54:32.8936367Z Running inductor/test_triton_heuristics 1/1 ... [2024-08-08 20:54:32.893274] 2024-08-08T20:54:32.8941981Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_triton_heuristics.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:32.893756] 2024-08-08T20:54:36.4652540Z 2024-08-08T20:54:36.4654945Z inductor/test_triton_heuristics 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_triton_heuristics_1.1_ed49934f6ed4d398_.log 2024-08-08T20:54:36.4656488Z 2024-08-08T20:54:36.4657645Z Running inductor/test_foreach 1/1 ... [2024-08-08 20:54:36.465466] 2024-08-08T20:54:36.4664374Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_foreach.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:36.465976] 2024-08-08T20:54:44.3470218Z 2024-08-08T20:54:44.3472220Z inductor/test_foreach 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_foreach_1.1_7344e9d442d5990d_.log 2024-08-08T20:54:44.3473579Z Running 0 items in this shard: 2024-08-08T20:54:44.3473953Z 2024-08-08T20:54:44.3475983Z Running inductor/test_inductor_freezing 1/1 ... [2024-08-08 20:54:44.347256] 2024-08-08T20:54:44.3482617Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_inductor_freezing.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:44.347755] 2024-08-08T20:54:52.1278678Z 2024-08-08T20:54:52.1280951Z inductor/test_inductor_freezing 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_inductor_freezing_1.1_a96117fa233b2d5c_.log 2024-08-08T20:54:52.1282432Z Running 0 items in this shard: 2024-08-08T20:54:52.1282806Z 2024-08-08T20:54:52.1283865Z Running inductor/test_fx_fusion 1/1 ... [2024-08-08 20:54:52.128065] 2024-08-08T20:54:52.1290144Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_fx_fusion.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:52.128545] 2024-08-08T20:54:57.0030030Z 2024-08-08T20:54:57.0032648Z inductor/test_fx_fusion 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_fx_fusion_1.1_59bf46ab3d7b2bdb_.log 2024-08-08T20:54:57.0034234Z Running 0 items in this shard: 2024-08-08T20:54:57.0034924Z 2024-08-08T20:54:57.0035747Z Running inductor/test_cpu_cpp_wrapper 1/13 ... [2024-08-08 20:54:57.003186] 2024-08-08T20:54:57.0041547Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_cpu_cpp_wrapper.py', '-m', 'serial', '--shard-id=1', '--num-shards=13', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:54:57.003648] 2024-08-08T20:55:05.0347910Z 2024-08-08T20:55:05.0350019Z inductor/test_cpu_cpp_wrapper 1/13 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_cpu_cpp_wrapper_1.13_d16863c939a3ed8f_.log 2024-08-08T20:55:05.0351344Z 2024-08-08T20:55:05.0353511Z Running inductor/test_cpu_cpp_wrapper 10/13 ... [2024-08-08 20:55:05.034999] 2024-08-08T20:55:05.0360067Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_cpu_cpp_wrapper.py', '-m', 'serial', '--shard-id=10', '--num-shards=13', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:55:05.035500] 2024-08-08T20:55:13.1672883Z 2024-08-08T20:55:13.1675012Z inductor/test_cpu_cpp_wrapper 10/13 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_cpu_cpp_wrapper_10.13_0f38537f8281c893_.log 2024-08-08T20:55:13.1676308Z 2024-08-08T20:55:13.1676812Z Running inductor/test_cpu_cpp_wrapper 11/13 ... [2024-08-08 20:55:13.166394] 2024-08-08T20:55:13.1679202Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_cpu_cpp_wrapper.py', '-m', 'serial', '--shard-id=11', '--num-shards=13', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:55:13.166898] 2024-08-08T20:55:21.3982246Z 2024-08-08T20:55:21.3984099Z inductor/test_cpu_cpp_wrapper 11/13 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_cpu_cpp_wrapper_11.13_5419d3188ef78d34_.log 2024-08-08T20:55:21.3985339Z 2024-08-08T20:55:21.4097206Z Running test_decomp 1/7 ... [2024-08-08 20:55:21.409265] 2024-08-08T20:55:21.4104089Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_decomp.py', '-m', 'not serial', '--shard-id=1', '--num-shards=7', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:55:21.409880] 2024-08-08T20:55:21.4106386Z Running test_model_exports_to_core_aten 1/1 ... [2024-08-08 20:55:21.409949] 2024-08-08T20:55:21.4110800Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_model_exports_to_core_aten.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:55:21.410535] 2024-08-08T20:55:25.9341606Z 2024-08-08T20:55:25.9344490Z test_model_exports_to_core_aten 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_model_exports_to_core_aten_1.1_da3fe7c3c7d7e941_.log 2024-08-08T20:55:25.9346825Z Running 1 items in this shard: test/test_model_exports_to_core_aten.py::TestQuantizePT2EModels::test_vit_aten_export 2024-08-08T20:55:25.9347775Z 2024-08-08T20:55:29.8215714Z Running test_transformers 1/1 ... [2024-08-08 20:55:29.820915] 2024-08-08T20:55:29.8220603Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_transformers.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:55:29.821424] 2024-08-08T20:59:55.1236389Z 2024-08-08T20:59:55.1237768Z PRINTING LOG FILE of test_transformers 1/1 (test/test-reports/test_transformers_1.1_44c310efaea94729_.log) 2024-08-08T20:59:55.1374530Z Test results will be stored in test-reports/python-pytest/test_transformers/test_transformers-fc7fcbdc7cb25791.xml 2024-08-08T20:59:55.1377090Z ============================= test session starts ============================== 2024-08-08T20:59:55.1379233Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-08T20:59:55.1380597Z cachedir: .pytest_cache 2024-08-08T20:59:55.1382863Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-08T20:59:55.1384477Z rootdir: /var/lib/jenkins/workspace 2024-08-08T20:59:55.1385485Z configfile: pytest.ini 2024-08-08T20:59:55.1387241Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-08T20:59:55.1389265Z collecting ... collected 45344 items 2024-08-08T20:59:55.1390448Z stepcurrent: Cannot find last run test, not skipping 2024-08-08T21:00:00.7154367Z Running 45344 items in this shard: test/test_transformers.py::TestSDPAPrivateUse1Only::test_fused_sdp_choice_privateuseone, test/test_transformers.py::TestSDPAPrivateUse1Only::test_scaled_dot_product_fused_attention_overrideable, test/test_transformers.py::TestSDPAPrivateUse1Only::test_scaled_dot_product_fused_attention_overrideable_backward, test/test_transformers.py::TestTransformersCUDA::test_bias_is_none_cuda, test/test_transformers.py::TestTransformersCUDA::test_decoder_only_layer_cuda, test/test_transformers.py::TestTransformersCUDA::test_decoder_padding_and_src_mask_bool_cuda, test/test_transformers.py::TestTransformersCUDA::test_disable_fastpath_cuda, test/test_transformers.py::TestTransformersCUDA::test_encoder_is_causal_cuda, test/test_transformers.py::TestTransformersCUDA::test_encoder_padding_and_src_mask_bool_cuda, test/test_transformers.py::TestTransformersCUDA::test_is_causal_gpu_cuda, test/test_transformers.py::TestTransformersCUDA::test_kpm_mask_trailing_column_with_nested_tensor_cuda, test/test_transformers.py::TestTransformersCUDA::test_mask_check_fastpath_cuda, test/test_transformers.py::TestTransformersCUDA::test_mha_native_args_nb_heads_1_bias_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_mha_native_args_nb_heads_1_bias_True_cuda, test/test_transformers.py::TestTransformersCUDA::test_mha_native_args_nb_heads_8_bias_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_mha_native_args_nb_heads_8_bias_True_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim2_key_padding_mask_dim1_bool_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim2_key_padding_mask_dim1_float32_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim2_key_padding_mask_dim_2_bool_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim2_key_padding_mask_dim_2_float32_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_2_key_padding_mask_dim1_bool_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_2_key_padding_mask_dim1_float32_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_2_key_padding_mask_dim_2_bool_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_2_key_padding_mask_dim_2_float32_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_3_key_padding_mask_dim1_bool_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_3_key_padding_mask_dim1_float32_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_3_key_padding_mask_dim_2_bool_cuda, test/test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_3_key_padding_mask_dim_2_float32_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_causal_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_causal_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_causal_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_causal_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_causal_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_causal_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_no_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_no_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_no_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_causal_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_causal_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_causal_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_causal_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_causal_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_causal_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_no_attn_mask_dropout_p_0_0_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_no_attn_mask_dropout_p_0_2_cuda, test/test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_no_attn_mask_dropout_p_0_5_cuda, test/test_transformers.py::TestTransformersCUDA::test_script_encoder_subclass_cuda, test/test_transformers.py::TestTransformersCUDA::test_script_mha_in_proj_weight_none_cuda, test/test_transformers.py::TestTransformersCUDA::test_self_attn_TxT_attn_mask_cuda, test/test_transformers.py::TestTransformersCUDA::test_train_with_is_causal_cuda, test/test_transformers.py::TestTransformersCUDA::test_train_with_pad_and_catch_error_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformer_bias_is_none_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_False_training_False_enable_nested_tensor_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_False_training_False_enable_nested_tensor_True_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_False_training_True_enable_nested_tensor_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_False_training_True_enable_nested_tensor_True_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_True_training_False_enable_nested_tensor_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_True_training_False_enable_nested_tensor_True_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_True_training_True_enable_nested_tensor_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_True_training_True_enable_nested_tensor_True_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_False_use_autocast_False_d_model_12_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_False_use_autocast_False_d_model_256_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_False_use_autocast_True_d_model_12_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_False_use_autocast_True_d_model_256_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_True_use_autocast_False_d_model_12_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_True_use_autocast_False_d_model_256_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_True_use_autocast_True_d_model_12_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_True_use_autocast_True_d_model_256_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_square_input_with_no_grad_False_training_False_enable_nested_tensor_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_square_input_with_no_grad_False_training_True_enable_nested_tensor_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_square_input_with_no_grad_True_training_False_enable_nested_tensor_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoder_square_input_with_no_grad_True_training_True_enable_nested_tensor_False_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_no_fastpath_with_hooks_nhead_3_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_no_fastpath_with_hooks_nhead_4_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_src_mask_nhead_1_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_src_mask_nhead_4_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_src_mask_nhead_8_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_subclass_cuda, test/test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_subclass_model_cuda, test/test_transformers.py::TestTransformersCUDA::test_with_nested_tensor_input_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_dispatch_fails_no_backend_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_atteention_large_bf16_nan_values_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_attention_fail_with_non_square_causal_attention_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_autocast_fp32_bfloat16_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_autocast_fp32_float16_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_193_dropout_p_0_0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_193_dropout_p_0_2_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_204_dropout_p_0_0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_204_dropout_p_0_2_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_256_dropout_p_0_0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_256_dropout_p_0_2_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_flash_fail_fp32_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_fused_kernels_nested_broadcasting_error_cases_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_fused_kernels_nested_broadcasting_requires_grad_failure_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_fused_kernels_seq_len_0_inputs_fused_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_attn_mask_present_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_broadcast_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_dim_3_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_head_dim_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_invalid_dtype_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_1_dimensional_inputs_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_1_dimensional_inputs_kernel1_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_1_dimensional_inputs_kernel2_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_datatypes_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_datatypes_kernel1_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_datatypes_kernel2_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_devices_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_devices_kernel1_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_devices_kernel2_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_last_dim_stride_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_sequence_lengths_kernel0_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_mem_efficient_fail_bfloat16_less_than_sm80_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_nested_fails_on_padding_head_dim_cuda, test/test_transformers.py::TestSDPAFailureModesCUDA::test_unaligned_tensors_cuda, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_0_bfloat16_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_0_float16_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_0_float32_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_0_float64_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_7_bfloat16_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_7_float16_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_7_float32_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_7_float64_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_0_bfloat16_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_0_float16_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_0_float32_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_0_float64_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_7_bfloat16_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_7_float16_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_7_float32_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_7_float64_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_attention_math_with_negative_scale_kernel0_cuda, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_bfloat16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float16, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float32, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float64, test/test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_with_inf_cuda, test/test_transformers.py::TestSDPACUDA::test_sdp_math_gradcheck_contiguous_inputs_False_cuda, test/test_transformers.py::TestSDPACUDA::test_sdp_math_gradcheck_contiguous_inputs_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_cudnn_attention_different_dk_dv_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_cudnn_attention_fail_d128_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_backwards_throws_determinism_warning_fused_kernel0_warn_only_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_backwards_throws_determinism_warning_fused_kernel0_warn_only_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_query_dense_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_seq_len_1_inputs_fused_kernel0_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_sdp_choice_type_dense_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_sdp_choice_type_nested_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_long_sequence_mask_float16_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_long_sequence_mask_float32_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_non_contig_mask_bug_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_non_contiguous_mask_float16_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_non_contiguous_mask_float32_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_pad_mask_float16_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_pad_mask_float32_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_backwards_determinism_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_mask_variants_mask_dim_1_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_mask_variants_mask_dim_2_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_mask_variants_mask_dim_3_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_mask_variants_mask_dim_4_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_accuracy_type_dense_fused_kernel0_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_accuracy_type_nested_fused_kernel0_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_type_dense_is_contiguous_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_type_dense_is_contiguous_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_type_nested_is_contiguous_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_type_nested_is_contiguous_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_choice_with_determinism_warn_only_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_choice_with_determinism_warn_only_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_False_is_causal_False_bfloat16_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_False_is_causal_False_float16_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_False_is_causal_True_bfloat16_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_False_is_causal_True_float16_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_True_is_causal_False_bfloat16_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_True_is_causal_False_float16_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_True_is_causal_True_bfloat16_cuda_bfloat16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_True_is_causal_True_float16_cuda_float16, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_mem_efficient_grad_against_math_contiguous_inputs_False_is_causal_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_mem_efficient_grad_against_math_contiguous_inputs_False_is_causal_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_mem_efficient_grad_against_math_contiguous_inputs_True_is_causal_False_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_mem_efficient_grad_against_math_contiguous_inputs_True_is_causal_True_cuda, test/test_transformers.py::TestSDPACudaOnlyCUDA::test_singelton_head_dim_stride_ne_1_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_LOWER_RIGHT_shape0_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_LOWER_RIGHT_shape1_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_LOWER_RIGHT_shape2_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_LOWER_RIGHT_shape3_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_UPPER_LEFT_shape0_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_UPPER_LEFT_shape1_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_UPPER_LEFT_shape2_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_UPPER_LEFT_shape3_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_LOWER_RIGHT_shape0_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_LOWER_RIGHT_shape1_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_LOWER_RIGHT_shape2_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_LOWER_RIGHT_shape3_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_UPPER_LEFT_shape0_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_UPPER_LEFT_shape1_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_UPPER_LEFT_shape2_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_UPPER_LEFT_shape3_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_is_causal_and_mask_fails_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_is_causal_equals_upper_left_shape0_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_is_causal_equals_upper_left_shape1_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_is_causal_equals_upper_left_shape2_cuda, test/test_transformers.py::TestAttnBiasCUDA::test_is_causal_equals_upper_left_shape3_cuda 2024-08-08T21:00:06.0386981Z 2024-08-08T21:00:06.0394378Z test_transformers.py::TestSDPAPrivateUse1Only::test_fused_sdp_choice_privateuseone [1/2] c++ -MMD -MF open_registration_extension.o.d -DTORCH_EXTENSION_NAME=custom_device_extension -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1013\" -I/var/lib/jenkins/workspace/test/cpp_extensions -isystem /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/include -isystem /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/include/TH -isystem /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/include/THC -isystem /opt/conda/envs/py_3.10/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++17 -g -c /var/lib/jenkins/workspace/test/cpp_extensions/open_registration_extension.cpp -o open_registration_extension.o 2024-08-08T21:00:06.0402471Z [2/2] c++ open_registration_extension.o -shared -L/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib -lc10 -ltorch_cpu -ltorch -ltorch_python -o custom_device_extension.so 2024-08-08T21:00:06.0406416Z SKIPPED [0.0009s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/132862 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:06.0409915Z test_transformers.py::TestSDPAPrivateUse1Only::test_scaled_dot_product_fused_attention_overrideable PASSED [0.0129s] [ 0%] 2024-08-08T21:00:06.0412054Z test_transformers.py::TestSDPAPrivateUse1Only::test_scaled_dot_product_fused_attention_overrideable_backward PASSED [0.0219s] [ 0%] 2024-08-08T21:00:06.0414155Z test_transformers.py::TestTransformersCUDA::test_bias_is_none_cuda PASSED [0.0057s] [ 0%] 2024-08-08T21:00:06.0416122Z test_transformers.py::TestTransformersCUDA::test_decoder_only_layer_cuda SKIPPED [0.0002s] (Fairseq not found) [ 0%] 2024-08-08T21:00:06.0418338Z test_transformers.py::TestTransformersCUDA::test_decoder_padding_and_src_mask_bool_cuda SKIPPED [0.0002s] (not supported on pre-3.11 Python) [ 0%] 2024-08-08T21:00:06.0420334Z test_transformers.py::TestTransformersCUDA::test_disable_fastpath_cuda PASSED [0.2906s] [ 0%] 2024-08-08T21:00:06.0422031Z test_transformers.py::TestTransformersCUDA::test_encoder_is_causal_cuda PASSED [0.0063s] [ 0%] 2024-08-08T21:00:06.0424083Z test_transformers.py::TestTransformersCUDA::test_encoder_padding_and_src_mask_bool_cuda SKIPPED [0.0003s] (not supported on pre-3.11 Python) [ 0%] 2024-08-08T21:00:06.0426509Z test_transformers.py::TestTransformersCUDA::test_is_causal_gpu_cuda SKIPPED [0.0002s] (Platform does not supposrt fused SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.0428809Z test_transformers.py::TestTransformersCUDA::test_kpm_mask_trailing_column_with_nested_tensor_cuda PASSED [0.0449s] [ 0%] 2024-08-08T21:00:06.0430598Z test_transformers.py::TestTransformersCUDA::test_mask_check_fastpath_cuda PASSED [0.0165s] [ 0%] 2024-08-08T21:00:06.0513500Z test_transformers.py::TestTransformersCUDA::test_mha_native_args_nb_heads_1_bias_False_cuda PASSED [0.0049s] [ 0%] 2024-08-08T21:00:06.0515784Z test_transformers.py::TestTransformersCUDA::test_mha_native_args_nb_heads_1_bias_True_cuda PASSED [0.0051s] [ 0%] 2024-08-08T21:00:06.0517736Z test_transformers.py::TestTransformersCUDA::test_mha_native_args_nb_heads_8_bias_False_cuda PASSED [0.0042s] [ 0%] 2024-08-08T21:00:06.0519673Z test_transformers.py::TestTransformersCUDA::test_mha_native_args_nb_heads_8_bias_True_cuda PASSED [0.0236s] [ 0%] 2024-08-08T21:00:06.0522302Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim2_key_padding_mask_dim1_bool_cuda PASSED [0.0402s] [ 0%] 2024-08-08T21:00:06.0525007Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim2_key_padding_mask_dim1_float32_cuda PASSED [0.0037s] [ 0%] 2024-08-08T21:00:06.0527449Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim2_key_padding_mask_dim_2_bool_cuda PASSED [0.0103s] [ 0%] 2024-08-08T21:00:06.0530121Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim2_key_padding_mask_dim_2_float32_cuda PASSED [0.0039s] [ 0%] 2024-08-08T21:00:06.0532543Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_2_key_padding_mask_dim1_bool_cuda PASSED [0.0037s] [ 0%] 2024-08-08T21:00:06.0535015Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_2_key_padding_mask_dim1_float32_cuda PASSED [0.0038s] [ 0%] 2024-08-08T21:00:06.0537588Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_2_key_padding_mask_dim_2_bool_cuda PASSED [0.0039s] [ 0%] 2024-08-08T21:00:06.0540052Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_2_key_padding_mask_dim_2_float32_cuda PASSED [0.0040s] [ 0%] 2024-08-08T21:00:06.0542450Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_3_key_padding_mask_dim1_bool_cuda PASSED [0.0036s] [ 0%] 2024-08-08T21:00:06.0545029Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_3_key_padding_mask_dim1_float32_cuda PASSED [0.0037s] [ 0%] 2024-08-08T21:00:06.0547493Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_3_key_padding_mask_dim_2_bool_cuda PASSED [0.0039s] [ 0%] 2024-08-08T21:00:06.0549988Z test_transformers.py::TestTransformersCUDA::test_multiheadattention_fastpath_attn_mask_attn_mask_dim_3_key_padding_mask_dim_2_float32_cuda PASSED [0.0040s] [ 0%] 2024-08-08T21:00:06.0552443Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_attn_mask_dropout_p_0_0_cuda PASSED [0.0089s] [ 0%] 2024-08-08T21:00:06.0554726Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_attn_mask_dropout_p_0_2_cuda PASSED [0.0045s] [ 0%] 2024-08-08T21:00:06.0557002Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_attn_mask_dropout_p_0_5_cuda PASSED [0.0044s] [ 0%] 2024-08-08T21:00:06.0559893Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_causal_attn_mask_dropout_p_0_0_cuda PASSED [0.0096s] [ 0%] 2024-08-08T21:00:06.0562320Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_causal_attn_mask_dropout_p_0_2_cuda PASSED [0.0050s] [ 0%] 2024-08-08T21:00:06.0564614Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_2D_causal_attn_mask_dropout_p_0_5_cuda PASSED [0.0050s] [ 0%] 2024-08-08T21:00:06.0567041Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_attn_mask_dropout_p_0_0_cuda PASSED [0.0042s] [ 0%] 2024-08-08T21:00:06.0569325Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_attn_mask_dropout_p_0_2_cuda PASSED [0.0046s] [ 0%] 2024-08-08T21:00:06.0571866Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_attn_mask_dropout_p_0_5_cuda PASSED [0.0045s] [ 0%] 2024-08-08T21:00:06.0574308Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_causal_attn_mask_dropout_p_0_0_cuda PASSED [0.0048s] [ 0%] 2024-08-08T21:00:06.0576661Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_causal_attn_mask_dropout_p_0_2_cuda PASSED [0.0051s] [ 0%] 2024-08-08T21:00:06.0579078Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_3D_causal_attn_mask_dropout_p_0_5_cuda PASSED [0.0049s] [ 0%] 2024-08-08T21:00:06.0583708Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_no_attn_mask_dropout_p_0_0_cuda SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131086 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:06.0590011Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_no_attn_mask_dropout_p_0_2_cuda SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131146 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:06.0596453Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_3D_input_dim_no_attn_mask_dropout_p_0_5_cuda SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131123 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:06.0600647Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_attn_mask_dropout_p_0_0_cuda PASSED [0.0043s] [ 0%] 2024-08-08T21:00:06.0602863Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_attn_mask_dropout_p_0_2_cuda PASSED [0.0046s] [ 0%] 2024-08-08T21:00:06.0605226Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_attn_mask_dropout_p_0_5_cuda PASSED [0.0041s] [ 0%] 2024-08-08T21:00:06.0607536Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_causal_attn_mask_dropout_p_0_0_cuda PASSED [0.0048s] [ 0%] 2024-08-08T21:00:06.0609906Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_causal_attn_mask_dropout_p_0_2_cuda PASSED [0.0046s] [ 0%] 2024-08-08T21:00:06.0612310Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_2D_causal_attn_mask_dropout_p_0_5_cuda PASSED [0.0047s] [ 0%] 2024-08-08T21:00:06.0614627Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_attn_mask_dropout_p_0_0_cuda PASSED [0.0040s] [ 0%] 2024-08-08T21:00:06.0616889Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_attn_mask_dropout_p_0_2_cuda PASSED [0.0041s] [ 0%] 2024-08-08T21:00:06.0619241Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_attn_mask_dropout_p_0_5_cuda PASSED [0.0042s] [ 0%] 2024-08-08T21:00:06.0621663Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_causal_attn_mask_dropout_p_0_0_cuda PASSED [0.0045s] [ 0%] 2024-08-08T21:00:06.0624078Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_causal_attn_mask_dropout_p_0_2_cuda PASSED [0.0047s] [ 0%] 2024-08-08T21:00:06.0626518Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_4D_causal_attn_mask_dropout_p_0_5_cuda PASSED [0.0046s] [ 0%] 2024-08-08T21:00:06.0631037Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_no_attn_mask_dropout_p_0_0_cuda SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/129853 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:06.0637471Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_no_attn_mask_dropout_p_0_2_cuda SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131107 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:06.0643836Z test_transformers.py::TestTransformersCUDA::test_scaled_dot_product_attention_4D_input_dim_no_attn_mask_dropout_p_0_5_cuda SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131179 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:06.0647757Z test_transformers.py::TestTransformersCUDA::test_script_encoder_subclass_cuda PASSED [0.4122s] [ 0%] 2024-08-08T21:00:06.0649603Z test_transformers.py::TestTransformersCUDA::test_script_mha_in_proj_weight_none_cuda PASSED [0.0353s] [ 0%] 2024-08-08T21:00:06.0651853Z test_transformers.py::TestTransformersCUDA::test_self_attn_TxT_attn_mask_cuda SKIPPED [0.0003s] (4D mask not supported yet - activate when 4D mask supported) [ 0%] 2024-08-08T21:00:06.0654024Z test_transformers.py::TestTransformersCUDA::test_train_with_is_causal_cuda PASSED [0.0509s] [ 0%] 2024-08-08T21:00:06.0656339Z test_transformers.py::TestTransformersCUDA::test_train_with_pad_and_catch_error_cuda SKIPPED [0.0015s] (test is slow; run with PYTORCH_TEST_WITH_SLOW to enable test) [ 0%] 2024-08-08T21:00:06.0658912Z test_transformers.py::TestTransformersCUDA::test_transformer_bias_is_none_cuda PASSED [0.0488s] [ 0%] 2024-08-08T21:00:06.0661066Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_False_training_False_enable_nested_tensor_False_cuda PASSED [0.0533s] [ 0%] 2024-08-08T21:00:06.0663649Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_False_training_False_enable_nested_tensor_True_cuda PASSED [0.0535s] [ 0%] 2024-08-08T21:00:06.0666093Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_False_training_True_enable_nested_tensor_False_cuda PASSED [0.0557s] [ 0%] 2024-08-08T21:00:06.0668502Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_False_training_True_enable_nested_tensor_True_cuda PASSED [0.0561s] [ 0%] 2024-08-08T21:00:06.0670942Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_True_training_False_enable_nested_tensor_False_cuda PASSED [0.0437s] [ 0%] 2024-08-08T21:00:06.0673527Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_True_training_False_enable_nested_tensor_True_cuda PASSED [0.0754s] [ 0%] 2024-08-08T21:00:06.0676068Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_True_training_True_enable_nested_tensor_False_cuda PASSED [0.0584s] [ 0%] 2024-08-08T21:00:06.0678579Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_batch_first_True_training_True_enable_nested_tensor_True_cuda PASSED [0.0566s] [ 0%] 2024-08-08T21:00:06.0681140Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_False_use_autocast_False_d_model_12_cuda PASSED [0.0748s] [ 0%] 2024-08-08T21:00:06.0683933Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_False_use_autocast_False_d_model_256_cuda PASSED [0.1240s] [ 0%] 2024-08-08T21:00:06.0686784Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_False_use_autocast_True_d_model_12_cuda PASSED [0.2279s] [ 0%] 2024-08-08T21:00:06.0689574Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_False_use_autocast_True_d_model_256_cuda PASSED [0.3528s] [ 0%] 2024-08-08T21:00:06.0692459Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_True_use_autocast_False_d_model_12_cuda PASSED [0.0636s] [ 0%] 2024-08-08T21:00:06.0695186Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_True_use_autocast_False_d_model_256_cuda PASSED [0.0880s] [ 0%] 2024-08-08T21:00:06.0697950Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_True_use_autocast_True_d_model_12_cuda PASSED [0.1593s] [ 0%] 2024-08-08T21:00:06.0700805Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_fastpath_use_torchscript_False_enable_nested_tensor_True_use_autocast_True_d_model_256_cuda PASSED [0.2902s] [ 0%] 2024-08-08T21:00:06.0703487Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_square_input_with_no_grad_False_training_False_enable_nested_tensor_False_cuda PASSED [0.0118s] [ 0%] 2024-08-08T21:00:06.0706079Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_square_input_with_no_grad_False_training_True_enable_nested_tensor_False_cuda PASSED [0.0113s] [ 0%] 2024-08-08T21:00:06.0708665Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_square_input_with_no_grad_True_training_False_enable_nested_tensor_False_cuda PASSED [0.0104s] [ 0%] 2024-08-08T21:00:06.0711251Z test_transformers.py::TestTransformersCUDA::test_transformerencoder_square_input_with_no_grad_True_training_True_enable_nested_tensor_False_cuda PASSED [0.0110s] [ 0%] 2024-08-08T21:00:06.0713626Z test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_no_fastpath_with_hooks_nhead_3_cuda PASSED [0.0036s] [ 0%] 2024-08-08T21:00:06.0715849Z test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_no_fastpath_with_hooks_nhead_4_cuda PASSED [0.0039s] [ 0%] 2024-08-08T21:00:06.0717879Z test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_src_mask_nhead_1_cuda PASSED [0.0054s] [ 0%] 2024-08-08T21:00:06.0719888Z test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_src_mask_nhead_4_cuda PASSED [0.0050s] [ 0%] 2024-08-08T21:00:06.0721944Z test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_src_mask_nhead_8_cuda PASSED [0.0050s] [ 0%] 2024-08-08T21:00:06.0724002Z test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_subclass_cuda PASSED [0.4428s] [ 0%] 2024-08-08T21:00:06.0725980Z test_transformers.py::TestTransformersCUDA::test_transformerencoderlayer_subclass_model_cuda PASSED [2.0999s] [ 0%] 2024-08-08T21:00:06.0727921Z test_transformers.py::TestTransformersCUDA::test_with_nested_tensor_input_cuda PASSED [0.0207s] [ 0%] 2024-08-08T21:00:06.0729753Z test_transformers.py::TestSDPAFailureModesCUDA::test_dispatch_fails_no_backend_cuda PASSED [0.0022s] [ 0%] 2024-08-08T21:00:06.0731975Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_atteention_large_bf16_nan_values_cuda SKIPPED [0.0003s] (Does not support flash attention) [ 0%] 2024-08-08T21:00:06.0734573Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_attention_fail_with_non_square_causal_attention_cuda SKIPPED [0.0002s] (Does not support flash attention) [ 0%] 2024-08-08T21:00:06.0737132Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_autocast_fp32_bfloat16_cuda SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.0739582Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_autocast_fp32_float16_cuda SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.0742378Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_193_dropout_p_0_0_cuda SKIPPED [0.0002s] (Does not support fused SDPA or not SM86+ hardware) [ 0%] 2024-08-08T21:00:06.0745223Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_193_dropout_p_0_2_cuda SKIPPED [0.0002s] (Does not support fused SDPA or not SM86+ hardware) [ 0%] 2024-08-08T21:00:06.0748111Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_204_dropout_p_0_0_cuda SKIPPED [0.0005s] (Does not support fused SDPA or not SM86+ hardware) [ 0%] 2024-08-08T21:00:06.0751110Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_204_dropout_p_0_2_cuda SKIPPED [0.0002s] (Does not support fused SDPA or not SM86+ hardware) [ 0%] 2024-08-08T21:00:06.0754054Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_256_dropout_p_0_0_cuda SKIPPED [0.0003s] (Does not support fused SDPA or not SM86+ hardware) [ 0%] 2024-08-08T21:00:06.0757022Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_backward_failure_sm86plus_head_dim_256_dropout_p_0_2_cuda SKIPPED [0.0002s] (Does not support fused SDPA or not SM86+ hardware) [ 0%] 2024-08-08T21:00:06.0760038Z test_transformers.py::TestSDPAFailureModesCUDA::test_flash_fail_fp32_cuda SKIPPED [0.0003s] (Does not support fused SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.0762314Z test_transformers.py::TestSDPAFailureModesCUDA::test_fused_kernels_nested_broadcasting_error_cases_cuda PASSED [0.0039s] [ 0%] 2024-08-08T21:00:06.0764877Z test_transformers.py::TestSDPAFailureModesCUDA::test_fused_kernels_nested_broadcasting_requires_grad_failure_cuda SKIPPED [0.0002s] (Fused SDPA was not built for this system) [ 0%] 2024-08-08T21:00:06.0767292Z test_transformers.py::TestSDPAFailureModesCUDA::test_fused_kernels_seq_len_0_inputs_fused_kernel0_cuda PASSED [0.0040s] [ 0%] 2024-08-08T21:00:06.0769558Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_attn_mask_present_kernel0_cuda SKIPPED [0.0002s] (Does not support flash attention) [ 0%] 2024-08-08T21:00:06.0771916Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_broadcast_kernel0_cuda PASSED [0.0019s] [ 0%] 2024-08-08T21:00:06.0774051Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_dim_3_kernel0_cuda PASSED [0.0070s] [ 0%] 2024-08-08T21:00:06.0776599Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_head_dim_kernel0_cuda SKIPPED [0.0003s] (Does not flash_attention fused scaled dot product attention) [ 0%] 2024-08-08T21:00:06.0779006Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_fused_inputs_invalid_dtype_kernel0_cuda PASSED [0.0017s] [ 0%] 2024-08-08T21:00:06.0781077Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_1_dimensional_inputs_kernel0_cuda PASSED [0.0015s] [ 0%] 2024-08-08T21:00:06.0783145Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_1_dimensional_inputs_kernel1_cuda PASSED [0.0015s] [ 0%] 2024-08-08T21:00:06.0785287Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_1_dimensional_inputs_kernel2_cuda PASSED [0.0015s] [ 0%] 2024-08-08T21:00:06.0787360Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_datatypes_kernel0_cuda PASSED [0.0015s] [ 0%] 2024-08-08T21:00:06.0789411Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_datatypes_kernel1_cuda PASSED [0.0015s] [ 0%] 2024-08-08T21:00:06.0791578Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_datatypes_kernel2_cuda PASSED [0.0015s] [ 0%] 2024-08-08T21:00:06.0793613Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_devices_kernel0_cuda PASSED [0.0015s] [ 0%] 2024-08-08T21:00:06.0795579Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_devices_kernel1_cuda PASSED [0.0015s] [ 0%] 2024-08-08T21:00:06.0797693Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_inputs_different_devices_kernel2_cuda PASSED [0.0016s] [ 0%] 2024-08-08T21:00:06.0799658Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_last_dim_stride_kernel0_cuda PASSED [0.0053s] [ 0%] 2024-08-08T21:00:06.0801571Z test_transformers.py::TestSDPAFailureModesCUDA::test_invalid_sequence_lengths_kernel0_cuda PASSED [0.0052s] [ 0%] 2024-08-08T21:00:06.0803584Z test_transformers.py::TestSDPAFailureModesCUDA::test_mem_efficient_fail_bfloat16_less_than_sm80_cuda PASSED [0.0054s] [ 0%] 2024-08-08T21:00:06.0805885Z test_transformers.py::TestSDPAFailureModesCUDA::test_nested_fails_on_padding_head_dim_cuda SKIPPED [0.0002s] (Fused SDPA was not built for this system) [ 0%] 2024-08-08T21:00:06.0808333Z test_transformers.py::TestSDPAFailureModesCUDA::test_unaligned_tensors_cuda SKIPPED [0.0002s] (Does not support fused SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.0810864Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_0_bfloat16_cuda_bfloat16 SKIPPED [0.0014s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0813141Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_0_float16_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0815435Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_0_float32_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0817833Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_0_float64_cuda_float64 SKIPPED [0.0017s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0820136Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_7_bfloat16_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0822656Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_7_float16_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0825007Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_7_float32_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0827298Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_dense_dropout_0_7_float64_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0829700Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_0_bfloat16_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0832036Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_0_float16_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0834413Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_0_float32_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0836723Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_0_float64_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0839044Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_7_bfloat16_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0841375Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_7_float16_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0843677Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_7_float32_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0845978Z test_transformers.py::TestSDPACUDA::test_fused_sdp_choice_cpu_type_nested_dropout_0_7_float64_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0848177Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_attention_math_with_negative_scale_kernel0_cuda PASSED [0.0023s] [ 0%] 2024-08-08T21:00:06.0851239Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0854954Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0858914Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_False_cuda_bfloat16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0862779Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0866447Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0870273Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0874162Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0877813Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0881526Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0885136Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0888756Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_False_cuda_float16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0892442Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0896071Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0899702Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0903413Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0907075Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float16_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0910644Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0914372Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0918079Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_False_cuda_float32 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0921791Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0925506Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0929121Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0932789Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0936416Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float32_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0940021Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0943709Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_0_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0947399Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_False_cuda_float64 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0951009Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_2_bool_mask_1_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0954754Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0958692Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_0_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0962330Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0966193Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_mask_vs_math_cpu_fused_kernel0_float64_batch_size_2_q_seq_len_267_kv_seq_len_514_n_head_3_head_dim_8_mask_dim_4_bool_mask_1_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0969799Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0973170Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0976604Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0979955Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0983304Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0986730Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0990062Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0993472Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.0996833Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1000194Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1003558Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1006919Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1010268Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1013670Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1017129Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1020569Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1023937Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1027285Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1030624Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1034022Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1037336Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1040608Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1044021Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1047329Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1050729Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1054153Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1057517Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1061314Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1064819Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1068281Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1071702Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1075058Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1078417Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1081789Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1085078Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1088512Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1091934Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1095322Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1098650Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1102051Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1105376Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1108711Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1112247Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1115665Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1118989Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1122411Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1125711Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1129081Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1132505Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1135836Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1139214Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0019s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1142638Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1145896Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1149225Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1152637Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1156018Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1161480Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1165088Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1168531Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1171922Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1175264Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1178590Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1181981Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1185294Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_bfloat16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1188557Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1192000Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1195310Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float16 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1198619Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1202079Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1205343Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1208641Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1212125Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1215538Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1218921Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1222249Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float16 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1225514Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1228883Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1232192Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1235509Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1238925Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1242244Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1245487Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1248854Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float16 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1252165Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1255513Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1259452Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1262896Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1266119Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1269550Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1272925Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1276320Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float16 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1279625Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1282917Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1286245Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1289538Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1292817Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1296206Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1299523Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1302821Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float16 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1306130Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1309595Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1313016Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1316383Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1319687Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1323044Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1326313Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1329629Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float16 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1332992Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1336322Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1339613Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1343000Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1346261Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1349543Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1352936Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1356356Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float16 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1360113Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1363468Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1366750Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1370035Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1373388Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1376669Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1380068Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1383322Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float16 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1386619Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1389969Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1393264Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1396571Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1399902Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float16_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float16 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1403163Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1407406Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1410966Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float32 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1414286Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1417630Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1421041Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1424278Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1427567Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1430990Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1434321Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float32 SKIPPED [0.0014s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1437736Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float32 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1441050Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1444308Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1447714Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1451024Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1454508Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1457991Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1461778Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1465053Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float32 SKIPPED [0.0019s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1468473Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1471775Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1475068Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1478471Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1481778Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1485008Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1488451Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float32 SKIPPED [0.0014s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1491762Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float32 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1495150Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1498461Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1501835Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1505332Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1508621Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1511912Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1515329Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1518654Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float32 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1521897Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1525344Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1528649Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1531950Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1535326Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1538626Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1541898Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1545271Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float32 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1548548Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1552001Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1555479Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1559169Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1562529Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1565857Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1569132Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1572500Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float32 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1575793Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1579076Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1582408Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1585696Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1588979Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1592367Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1595696Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1599194Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float32 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1602568Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1605851Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1609203Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1612485Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1615774Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float32_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float32 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1619233Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1622516Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1625843Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float64 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1629279Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1632601Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1635905Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1639256Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1642544Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1645928Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1649422Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1652733Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float64 SKIPPED [0.0015s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1656163Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1659823Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1663166Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1666588Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1669895Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1673199Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1676588Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1679856Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float64 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1683136Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1686523Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1689815Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1693111Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1696600Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1699998Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1703298Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1706693Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float64 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1710036Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1713378Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1716688Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1719913Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1723277Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_12_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1726597Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1729901Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1733285Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_False_cuda_float64 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1736636Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_16_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1739906Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1743360Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1746726Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1750024Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_1_head_dim_8_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1753393Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1756757Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1760465Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_False_cuda_float64 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1763868Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_16_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1767162Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1770543Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1773872Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1777155Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_1030_n_head_3_head_dim_8_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1780444Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1783761Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1787050Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_False_cuda_float64 SKIPPED [0.0019s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1790398Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_16_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1793843Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1797188Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1800538Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1803828Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_1_head_dim_8_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1807156Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1810541Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1813825Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_False_cuda_float64 SKIPPED [0.0016s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1817042Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_16_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1820426Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1823698Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_False_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1827039Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_False_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1830327Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_vs_math_cpu_fused_kernel0_float64_batch_size_2_seq_len_267_n_head_3_head_dim_8_causal_True_train_True_cuda_float64 SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1832978Z test_transformers.py::TestSDPACUDA::test_scaled_dot_product_fused_attention_with_inf_cuda SKIPPED [0.0013s] (Only runs on cpu) [ 0%] 2024-08-08T21:00:06.1837287Z test_transformers.py::TestSDPACUDA::test_sdp_math_gradcheck_contiguous_inputs_False_cuda SKIPPED [0.0008s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131625 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:06.1843295Z test_transformers.py::TestSDPACUDA::test_sdp_math_gradcheck_contiguous_inputs_True_cuda SKIPPED [0.0010s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131255 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:06.1847561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_cudnn_attention_different_dk_dv_cuda SKIPPED [0.0003s] (cuDNN Attention is not supported on this system) [ 0%] 2024-08-08T21:00:06.1850142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_cudnn_attention_fail_d128_cuda SKIPPED [0.0002s] (cuDNN Attention is not supported on this system) [ 0%] 2024-08-08T21:00:06.1853291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1857087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1861313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1864958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1868669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1872476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1876174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1879923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1883661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1887356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1891259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1895089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1898769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1902581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1906283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1909932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1913676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 0%] 2024-08-08T21:00:06.1917378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1921082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1924864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1928543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1932331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1936024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1939816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1943584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1947289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1950986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1954813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1958991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1962702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1966481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1970187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1973909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1977976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1981708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1985488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1989347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1993107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.1996896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2000581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2004300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2008127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2011841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2015478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2019248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2022943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2026646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2030405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2034100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2037948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2041713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2045388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2049140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2052872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2056610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2060811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2064557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2068208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2071988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2075680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2079362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2083159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2086973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2090826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2094459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2098152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2101914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2105585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2109287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2113072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2116703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2120372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2124166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2127886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2131579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2135430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2139237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2142892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0010s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2146682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2150369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2154188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2157940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2162113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2165879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2169518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2173211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2176999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2180656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2184492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2188424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2192119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2195739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2199556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2203267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2207014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2210707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2214410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2218130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2221856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2225544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2229316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2233112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2236936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2240715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2244353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2248079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2251864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2255602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2259825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2263838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2267561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2271247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2274946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2278658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2282605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2286408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2290095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2293909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2297568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2301232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2304997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2308712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2312459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2316156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2319887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2323601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2327314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2331069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2334952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2338666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2342369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2346105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2349787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2353482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2357322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2361449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2365236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2368949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2372575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2376293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2380131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2383950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2387697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2391386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2395110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2398841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2402560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2406240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2410019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2413738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2417519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2421232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2424872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2428757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2432558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2436288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2440066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2443781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2447405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2451134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2454888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2458964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2462804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2466517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2470257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2473913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2477766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2481632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2485337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2489031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2492804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2496471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2500107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2503903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0010s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2507584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2511249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2515069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2518773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2522527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2526249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2530023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2533777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2537455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2541126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2544859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2548512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2552204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2556037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2560096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2563833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2567620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2571285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2575043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2578983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2582641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2586377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2590081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2593825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2597610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2601239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2604924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2608726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2612428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2616107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2619870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2623657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2627363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2631143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2634832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2638592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2642297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2645957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2649655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2653376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2657099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2661334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2665020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2668713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2672658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2676390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2680087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2683884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2687573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2691335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2695053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2698778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2702487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2706173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2709866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2713663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2717326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2721066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2724910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2728567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2732236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2736007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2739686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2743482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2747186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2750820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2754609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2758678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2762386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2766161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2769978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2773805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2777510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2781189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2784877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2788670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2792366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2796102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2799782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2803412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2807142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2810788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2814467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2818314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2822081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2825722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2829496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2833185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2836843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2840606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2844292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2848046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2851691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2855414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2859557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2863306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2867186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2871091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2874793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2878451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2882245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2885926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2889598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2893375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2897064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2900794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2904465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2908175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2911943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2915719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2919486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2923273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2926956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2930574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2934359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2938051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2941827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2945616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2949334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2952996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2956808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2960964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2964902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2968751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2972443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2976190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2979850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2983526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2987313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2990969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2994630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.2998432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3002113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3005710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3009494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3013277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3017142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3020828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3024556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3028336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3031966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3035641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3039413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3043118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3046818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3050570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3054208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3057955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3062290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3066121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3069882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3073604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3077274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3080941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3084634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3088342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3092099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3095761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3099458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3103271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3106892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3110652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3114515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3118212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3121954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3125646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3129326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3133047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3136775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3140430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3144244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3147973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3151642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3155390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3159596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3163402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3167205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3170843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3174610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3178311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3181954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3185708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3189382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3193089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3196832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3200494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3204160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3207975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3211760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3215428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3219264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3222997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3226760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3230420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3234116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3237909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3241631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3245306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3249087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3252768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3256512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3260812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3264494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3268196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3272007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3275675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3279452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3283122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3286799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3290547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3294368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3297945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3301752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3305565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3309319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3313132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3316839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3320519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3324319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3328020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3331668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3335453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3339173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3343010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3346696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3350351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3354182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3357974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3362070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3365845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3369528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3373223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3377014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3380708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3384316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3388077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3391763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3395469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3399172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3402995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3406866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3410490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3414172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3417992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3421673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3425336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3429104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3432718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3436373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3440092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3443763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3447517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3451290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3455110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3459207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3462970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3466657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3470382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3474068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3477775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3481538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3485162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3488846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3492659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3496367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3500282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3504125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3507825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3511559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3515251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3518926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3522688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3526373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3530033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3533722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3537444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3541400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3545163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3548951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3552809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3556563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3560637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3564419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3568129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3571799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3575563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3579261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3582945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3586652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3590322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3594004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3597921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3601718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3605457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3609149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3612808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 1%] 2024-08-08T21:00:06.3616542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3620201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3623887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3627642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3631385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3634904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3638681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3642365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3646150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3650071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3653728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3657552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3661643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3665303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3669115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3672871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3676544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3680282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3683979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3687637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3691407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3695275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3699075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3702824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3706500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3710082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3713819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3717515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3721239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3724888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3728570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3732349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3735991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3739624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3743453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3747213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3750870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3754681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3758721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3762390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3766195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3769908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3773656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3777359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3781033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3784776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3788394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3792224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3796101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3799759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3803431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3807239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3810863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3814511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3818256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3821939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3825659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3829331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3833015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3836755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3840528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3845050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3848869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3852558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3856303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3860459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3864100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3867808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3871576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3875218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3878938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3882620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3886210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3890082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3893875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3897615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3901268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3904942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3908587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3912273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3915941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3919577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3923325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3927049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3930785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3934458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3938163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3942032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3945709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3949388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3953140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3956890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3960963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3964752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3968473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3972148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3976082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3979753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3983491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3987268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3991059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3994789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.3998484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4002163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4005997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4009668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4236329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4239979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4243521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4247065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4250624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4254185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4258150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4262331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4265881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4269454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4273007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4276540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4280074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4283616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4287153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4290667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4294219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4297789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4301334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4304983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4308642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4312209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4315771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4319314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4322869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4326448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4330010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4333554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4337109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4340702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4344278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4347824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4351471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4355197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4359138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4362737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4366297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4369860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4373401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4376927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4380512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4384056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4387617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4391171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4394731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4398451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4402135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4405675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4409237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4412817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4416356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4419887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4423442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4427013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4430565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4434116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4437680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4441250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4444897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4448509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4452064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4455658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4459627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4463190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4466737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4470299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4473925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4477371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4480920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4484584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4488032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4491698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4495478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4498953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4502527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4506093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4509667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4513246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4516818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4520388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4523946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4527529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4531112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4534667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4538308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4541957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4545501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4549026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4552594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4556218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4560148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4563731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4567284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4570843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4574415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4577964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4581518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4585261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4588922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4592467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4596020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4599621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4603186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4606746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4610324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4613899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4617460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4621005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4624574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4628129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4631785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4635416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4638975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4642556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4646111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4649652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4653215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4656856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4660831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4664381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4667925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4671516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4675060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4678735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4682432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4686019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4689581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4693115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4696702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4700260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4703834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4707378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4710930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4714489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4718016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4721547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4725173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4728837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4732379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4735915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4739476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4743016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4746575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4750116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4753662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4757282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4761398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4764947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4768517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4772261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4775955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4779512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4783092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4786688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4790243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4793805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4797371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4800928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4804478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4808017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4811584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4815145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4818787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4822424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4825977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4829558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4833126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4836662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4840211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4843781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4847330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4850867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4854425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4858067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4861992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4865665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4869350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4872928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4876500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4880060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4883607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4887150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4890696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4894236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4897775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4901346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4904905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4908436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4912056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4915727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4919274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4922811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4926371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4929931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4933484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4937006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4940562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4944139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4947713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4951266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4954819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4958898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4962613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4966165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4969721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4973380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4976817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4980343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4983876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4987440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4990981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4994527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.4998085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5001649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5005277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5008884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5012441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5016008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5017869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5019704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5021572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5023446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5025274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5027125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5028973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5030857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5032691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5034630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5036551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5038409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5040229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5042087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5043925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5045800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5047629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5049476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5051307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5053177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5055047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5056905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5059223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5061284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5063121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5064957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5066821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5068671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5070524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5072363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5074222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5076072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5077920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5079762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5081602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5083543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5085433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5087255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5089115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5090966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5092780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5094634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5096473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5098329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5100141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5102053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5104075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5105965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5107894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5109840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5111681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5113561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5115398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5117249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5119096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5120978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5122807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5124642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5126491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5128341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5130171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5132071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5134020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5135873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5137722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5139564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5141419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5143268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5145116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5146955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5148814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5150664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5152512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5154348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5156330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5158296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5160507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5162377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5164231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5166110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5167945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5169898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5171732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5173589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5175409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5177262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5179093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5181083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5183020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5184881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5186722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5188576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5190417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5192255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5194117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5195968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5197809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5199647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5201525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5203383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5205326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5207289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5209126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5210993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5212876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5214691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5216545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5218390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5220221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5222048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5223901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5225782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5227599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5229528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5231466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5233336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5235163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5237036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5238871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5240735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5242559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5244409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5246240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5248114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5249953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5251804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5253717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5255700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5257615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5259986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 2%] 2024-08-08T21:00:06.5261928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5263762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5265590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5267419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5269272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5271110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5272944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5274782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5276625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5278591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5280557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5282384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5284248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5286110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5287936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5289791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5291651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5293528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5295358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5297217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5299075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5300958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5302901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5304843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5306677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5308544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5310374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5312221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5314069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5315948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5317770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5319602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5321456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5323314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5325154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5327100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5329022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5330868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5332726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5334565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5336429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5338297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5340162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5342005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5343866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5345770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5347741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5349632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5351561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5353515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5355373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5357317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5359647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5361587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5363416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5365278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5367113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5368976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5370798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5372657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5374549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5376653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5378740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5380488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5382329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5384200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5386026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5387868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5389730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5391578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5393409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5395246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5397096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5398920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5400914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5402877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5404797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5406654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5408491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5410318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5412222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5414110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5416049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5417878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5419720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5421598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5423457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5425500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5427497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5429377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5431209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5433078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5434918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5436797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5438655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5440489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5442320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5444185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5446009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5447860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5449778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5451713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5453549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5455430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5457301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5459530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5461407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5463254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5465103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5466941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5468780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5470623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5472477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5474458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5476416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5478249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5480106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5481963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5483789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5485646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5487474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5489331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5491227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5492984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5494844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5496698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5498589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5500531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5502358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5504222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5506050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5507895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5509725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5511587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5513406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5515229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5517081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5518941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5520788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5522698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5524638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5526484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5528331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5530170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5532011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5533839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5535674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5537501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5539350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5541194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5543035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5544859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5546767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5548711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5550525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5552378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5554222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5556125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5557942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5560143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5562016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5563888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5565721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5567588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5569426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5571419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5573360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5575213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5577043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5578908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5580716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5582538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5584398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5586252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5588089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5589911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5591770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5593613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5595554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5597485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5599340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5601183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5603023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5604849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5606698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5608546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5610410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5612229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5614072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5615944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5617757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5619679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5621580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5623427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5625238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5627089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5628916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5630769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5632579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5634428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5636253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5638125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5639936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5641775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5643691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5645622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5647457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5649290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5651157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5653022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5654866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5656755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5658960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5660890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5662750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5664596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5666440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5668403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5670346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5672174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5674024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5675880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5677705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5679554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5681388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5683298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5685076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5686927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5688770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5690630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5692530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5694459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5696294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5698152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5699971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5701833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5703664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5705528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5707356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5709178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5711014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5712850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5714674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5716563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5718493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5720327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5722153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5723977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5725825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5727666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5729493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5731320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5733164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5735006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5736849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5738671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5740509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5742452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5744352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5746202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5748034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5749899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5751714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5753563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5755427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5757276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5759436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5761312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5763140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5764992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5767666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5769671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5771496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5773337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5775169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5776992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5778837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5780688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5782514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5784339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5786191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5788027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5789857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5791759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5793694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5795532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5797383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5799193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5801008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5802853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5804664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5806495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5808311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5810166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5811981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5813818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5815721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5817662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5819468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5821314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5823152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5825010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5826832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5828682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5830523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5832364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5834202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5836042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5837890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5839812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5841731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5843564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5845406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5847245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5849087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5850888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5852742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5854577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5856454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5858287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5860515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5862347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5864286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5866261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5868095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5869961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5871780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5873613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5875443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5877310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5879136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5880981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5882818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5884684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5886492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5888415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5890317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5892160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5893968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5895791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5897638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5899475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5901308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5903135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5905010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5906818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5908712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5910527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5912428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5914346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5916169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5917985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5919830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5921668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5923496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5925324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5927157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5929010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5930826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5932672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5934491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5936410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5938293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5940117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5941929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5943784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5945593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5947428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5949251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5951106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5952909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5954744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5956624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5958796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5960869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5962820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5964686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5966518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5968359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5970200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5972058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5973929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5975760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5977588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5979432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5981373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5983085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5985023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5986976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5988811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5990617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5992458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5994290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5996137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5997950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.5999794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6001616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6003472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6005290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6007130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6008954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6010893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6012791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6014635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6016469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6018330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6020141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6021963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6023803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6025626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6027446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6029266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6031104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6032937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6034844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6036746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6038577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6040410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6042241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6044053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6045905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6047747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6049572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6051384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6053216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6055080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6056935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6059269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6061245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6063099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6064919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6066762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6068574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6070419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6072230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6074050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6075869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6077720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6079529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6081338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6083250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6085158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6086978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6088801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6090647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6092480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6094303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6096204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6097987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6101574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6105179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6108749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6112163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 3%] 2024-08-08T21:00:06.6115789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6119399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6122913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6126439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6129976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6133511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6136952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6140495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6144037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6147550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6151060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6154563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6158156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6162194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6165853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6169368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6173019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6176537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6180065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6183578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6187135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6190660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6194194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6197790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6201247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6204767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6208364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6211961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6215479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6218976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6222454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6225969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6229500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6233106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6236613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6240128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6243658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6247158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6250656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6254176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6257847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6261894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6265407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6268949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6272489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6276016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6279547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6283088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6286636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6290170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6293681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6297196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6300717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6304364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6307975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6311497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6315031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6318543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6322047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6325569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6329121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6332632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6336150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6339707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6343261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6346815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6350455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6354123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6357748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6361700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6365248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6368813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6372403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6375983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6379519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6383065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6386631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6390145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6393673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6397343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6401036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6404591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6408128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6411694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6415249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6418821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6422371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6425919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6429498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6433060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6436603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6440143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6443846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6447504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6451053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6454620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6458259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6462169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6465739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6469312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6472874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6476415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6479954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6483501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6487049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6490737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6494413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6497966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6501538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6505106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6508651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6512195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6515762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6519299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6522836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6526393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6529961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6533615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6537124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6540757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6544327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6547880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6551430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6554962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6558879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6562499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6566016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6569564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6573149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6576681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6580206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6583849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6587525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6591069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6594602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6598167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6601761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6605307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6608840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6612399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6615983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6619544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6623114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6626672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6630342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6633979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6637524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6641082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6644641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6648182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6651709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6655241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6659198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6662782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6666589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6670171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6673745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6677438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6681101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6684654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6688224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6691794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6695325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6698901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6702495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6706038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6709576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6713148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6716733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6720293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6723922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6727565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6731096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6734665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6738180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6741727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6745295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6748855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6752373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6755965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6759905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6763496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6767028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6770713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6774391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6777912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6781445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6785004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6788571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6792145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6795690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6799244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6802790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6806348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6809899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6813432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6817065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6820685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6824198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6827728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6831298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6834846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6838377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6841926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6845479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6849011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6852556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6856119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6860089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6863815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6867476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6871039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6874607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6878187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6881737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6885297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6888893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6892459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6896013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6899552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6903142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6906670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6910292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6913927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6917488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6921035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6924585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6928137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6931700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6935264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6938794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6942338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6945889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6949442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6952974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6956654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6960690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6964252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6967797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6971341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6974905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6978459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6981998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6985534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6989076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6992610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6996121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.6999659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7003337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7006991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7010508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7014052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7017607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7021147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7024675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7028221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7031775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7035316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7038847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7042388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7045958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7049612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7053245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7056831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7060787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7064356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7067880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7071422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7074969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7078498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7082014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7085562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7089135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7092660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7096312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7099980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7103541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0004s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7107088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7110618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7114142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7117719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7121266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7124793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7128339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7131919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0004s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7135489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7139019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7142674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7146331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7149878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7153416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7157004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7161002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7164519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7168053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7171593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7175147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7178681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7182206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7185728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7189447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7193148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7196658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7200193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7203753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7207279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7210780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7214329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7217888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7221432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7224964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7228511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7232080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7235684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7239293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7242813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7246360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7249885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7253393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7256941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7260893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7264447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7267964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7271503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7275053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7278574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7282192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7285849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7289408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7292975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7296485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7300028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7303603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0009s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7307143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7310677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7314236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7317806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7321383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7324892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7328521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7332125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7335658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7339187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7342747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7346308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7349847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7353365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7356950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7360866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7364425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7367957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7371503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7375210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7378864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7382383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7385911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7389497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7393061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7396589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7400157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7403713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7407270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7410806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7414359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7417896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7421507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7425082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7428632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7432194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7435746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7439293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7442834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7446401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7449928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7453453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7457048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7461108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7464656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7468337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7472028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7475582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7479148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7482694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7486260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7489828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7493388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7496934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7500457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7504005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7507875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7511434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7515084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7518730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7522238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7525757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7529287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7532851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7536375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7539896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7543432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7546963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7550502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7554023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7557609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7561745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0005s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7565435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7568963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7572486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7576052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7579600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7583125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7586666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7590206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7593698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7597217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7600777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7604296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7607931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7611526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7615056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7618582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7622115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7625648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7629183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7632755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7636313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7639850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7643386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7646964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7650518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7654057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7657782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7661874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7665440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7668971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7672529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7676079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7679605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7683132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7686665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7690204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7693752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7697278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7700815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7704540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7708220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7711746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7715274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7718847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7722382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7737451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7741067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7744633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7748205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 4%] 2024-08-08T21:00:06.7751761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7755318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7759434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7763298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7766976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7770518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7774086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7777609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7781112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7784654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7788213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7791753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7795291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7798839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7802400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7805925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7809548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7814100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7817652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7821187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7824723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7828262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7831812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7835371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7838909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7842459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7846031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7849570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7853079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7856747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7860852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7864376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7867895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7871438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7874983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7878497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7882034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7885557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7889101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7892633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7896161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7899696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7903398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7905381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7907212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7909076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7910939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7912787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7914627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7916495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7918341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7920173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7922008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7923862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7925697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7927606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7929519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7931350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7933212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7935031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7936871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7938707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7940579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7942398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7944239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7946071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7947936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7949742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7951659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7953573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7955437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7957302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7959543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7961407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7963257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7965107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7966935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7968769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7970599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7972426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7974242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7976228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7978188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7980018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7981844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7983705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7985537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7987366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7989195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7991051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7992888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7994704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7996552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.7998381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8000332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8002341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8004090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8005929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8007801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8009616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8011482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8013279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8015140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8016942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8018783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8020620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8022481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8024371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8026278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8028121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8029963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8031801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8033622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8035466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8037308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8039151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8040974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8042833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8044685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8046529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8048441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8050375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8052214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8054063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8055942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8057765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8060035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8061884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8063735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8065566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8067427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8069246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8071083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8073061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8075043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8076848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8078685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8080519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8082380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8084189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8086023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8087853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8089696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8091529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8093368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8095236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8097153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8099070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8100888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8102718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8104542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8106371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8108177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8110018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8111860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8113684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8115501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8117345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8119180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8120982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8122895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8124815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8126670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8128488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8130337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8132177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8134040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8135868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8137716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8139549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8141415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8143230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8145081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8146975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8148913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8150713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8152537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8154396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8156275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8158117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8160372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8162235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8164090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8165914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8167735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8169580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8171576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8173530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8175357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8177221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8179072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8180907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8182748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8184675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8186452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8188274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8190128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8191945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8193787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8195671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8197598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8199430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8201297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8203118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8204955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8206784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8208649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8210458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8212275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8214123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8215969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8217798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8219704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8221645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8223490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8225333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8227174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8229100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8231108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8232974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8234800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8236627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8238451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8240284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8242104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8244047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8245986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8247800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8249660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8251475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8253326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8255138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8257030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8259238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8261127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8262945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8264784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8266610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8268604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8270571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8272411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8274252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8276100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8277931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8279747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8281587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8283440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8285340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8287059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8288911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8290747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8292649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8294554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8296388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8298220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8300051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8301865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8303708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8305551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8307375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8309213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8311052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8312926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8314747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8316677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8318591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8320457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8322272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8324122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8325953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8327794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8329605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8331446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8333270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8335145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8336949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8338771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8340687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8342604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8344429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8346247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8348095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8349940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8351781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8353598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8355463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8357355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8359569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8361493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8363345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8365344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8367299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8369126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8370939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8372786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8374595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8376444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8378264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8380129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8381937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8383773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8385603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8387456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8389346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8391271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8393092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8394937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8396750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8398593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8400417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8402261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8404100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8405921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8407770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8409609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8411439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8413262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8415199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8417117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8418932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8420739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8422590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8424421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8426249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8428075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8429908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8431739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8433555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8435391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8437215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8439154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8441057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8442897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8444731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8446599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8448416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8450267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8452101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8453966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8455777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8457676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8459890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8461764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8463724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8465666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8467514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8469356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8471191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8473006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8474858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8476705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8478533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8480382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8482208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8484050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8485878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8487773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8489704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8491543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8493386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8495217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8497041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8498903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8500728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8502569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8504381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8506233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8508040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8509859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8511762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8513696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8515503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8517341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8519173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8521017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8522817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8524654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8526484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8528312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8530141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8531979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8533824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8535747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8537683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8539509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8541381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8543206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8545041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8546857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8548691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8550525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8552339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8554149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8556019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8557886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8560181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8562163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8563978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8565821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8567629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8569470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8571298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8573150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8574961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8576797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8578629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8580491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8582305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8584223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8586132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8587971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8589800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8591624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8593461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8595282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8597111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8598925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8600763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8602598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8604427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8606238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8608149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8610066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8611880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8613688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8615528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8617360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8619160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8620997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8622825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8624667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8626486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8628359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8630158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8632010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8633908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8635840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8637649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8639485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8641286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8643108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8644925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8646776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8648568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8650375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8652214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8654064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8655861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8657797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8660163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8662032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8663871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8665696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 5%] 2024-08-08T21:00:06.8667538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8669381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8671218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8673045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8674885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8676726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8678546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8680383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8682351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8684313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8686117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8687951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8689780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8691636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8693448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8695295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8697115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8698961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8700765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8702608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8704453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8706406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8708315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8710163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8712032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8713910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8715758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8717594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8719462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8721330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8723183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8725032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8726894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8728737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8730657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8732576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8734430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8736286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8738145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8739982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8741832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8743689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8745526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8747379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8749225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8751113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8752935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8754873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8756842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8759074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8760953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8762820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8764664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8766537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8768372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8770236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8772065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8773927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8775754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8777584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8779567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8781536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8783378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8785214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8787076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8788926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8790769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8792611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8794471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8796312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8798149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8799984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8801838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8803763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8805693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8807538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8809371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8811246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8813067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8814918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8816748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8818624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8820410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8822251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8824091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8825955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8827869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8829850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8831675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8833540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8835365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8837214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8839053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8840918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8842773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8844608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8846487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8848358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8850206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8852120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8854074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8855930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8857834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8860031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8861911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8863750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8865585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8867430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8869283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8871136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8872966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8874816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8876777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8878778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8880609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8882464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8884318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8886192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8888013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8889874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8891734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8893609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8895444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8897308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8899155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8901109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8903039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8904898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8906734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8908586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8910422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8912246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8914110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8915973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8917817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8919655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8921517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8923365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8925280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8927197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8929051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8930904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8932752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8934595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8936462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8938290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8940124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8941979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8943829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8945708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8947531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8949465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8951378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8953231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8955045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8956946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8959156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8961089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8962923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8964781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8966610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8968475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8970309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8972135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8974130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8976093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8977934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8979776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8981646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8983508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8985350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8987201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8989059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8990912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8992764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8994611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8996458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.8998376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9000314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9002140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9003982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9005857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9007684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9009536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9011375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9013247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9015069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9016923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9018763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9020624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9022519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9024448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9026285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9028155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9029987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9031842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9033700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9035535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9037383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9039218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9041059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9042900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9044725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9046625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9048559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9050403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9052238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9054070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9055915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9057794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9060000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9061867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9063721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9065566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9067387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9069227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9071192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9073183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9075009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9076868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9078710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9080580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9082402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9084258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9086095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9087941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9089753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9091607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9093442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9096093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9098050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9099875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9101702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9103555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9105404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9107232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9109090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9110944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9112789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9114621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9116487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9118342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9120264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9122185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9124044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9125898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9127747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9129586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9131427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9133261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9135080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9136920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9138764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9140638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9142450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9144372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9146280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9148142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9149963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9151818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9153655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9155517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9157376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9159856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9161742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9163615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9165450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9167285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9169281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9171241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9173080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9174895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9176733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9178569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9180388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9182214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9184065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9185900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9187730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9189560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9191402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9193331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9195245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9197060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9198893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9200758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9202583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9204423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9206263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9208138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9209959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9211818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9213664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9215535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9217353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9219285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9221225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9223070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9224885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9226741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9228583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9230413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9232252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9234074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9235923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9237771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9239605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9241439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9243366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9245295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9247128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9248960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9250823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9252667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9254522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9256412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9258272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9260557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9262403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9264256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9266079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9268059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9270000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9271841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9273681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9275553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9277369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9279217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9281052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9282912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9284736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9286597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9288428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9290285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9292185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9294097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9295951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9297812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9299660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9301495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9303359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9305223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9307060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9308896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9310741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9312572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9314389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9316287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9318223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9320067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9321906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9323744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9325573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9327438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9329262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9331114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9332950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9334818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9336633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9338469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9340406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9342331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9344152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9346007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9347849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9349717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9351536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9353385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9355208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9357079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9359274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9361137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9362985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9364943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9366906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9368724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9370581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9372424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9374245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9376066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9377945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9379779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9381611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9383447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9385288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9387142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9389045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9390984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9392823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9394695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9396534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9398388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9400214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9402075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9403879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9405717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9407553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9409432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9411244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9413167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9415102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9416936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9418758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9420591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9422435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9424283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9426124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9427964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9429800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9431654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9433510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9435345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9437278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9439217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9441056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9442894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9444741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9446574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9448407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9450237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9452069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9453927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9455739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9457636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9459849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9461885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9463842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9465691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9467529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9469398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9471214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9473053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9474896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9476770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9478610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9480448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9482295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9484138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9486054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9487962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9489796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9491658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9493462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9495282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9497129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9498983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9500823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9502647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9504499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9506348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9508169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9510069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9512017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9513862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 6%] 2024-08-08T21:00:06.9515685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9517543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9519378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9521245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9523081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9524953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9526795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9528676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9530518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9532364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9534265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9536201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9538020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9539854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9541701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9543566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9545378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9547211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9549063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9550911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9552751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9554590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9556482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9558794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9560851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9562690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9564554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9566408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9568242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9570075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9571930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9573782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9575616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9577453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9579278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9581125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9583012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9584936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9586769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9588634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9590452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9592301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9594130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9595991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9597807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9599646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9601487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9603357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9605169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9607132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9609062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9610908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9612742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9614603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9616445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9618287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9620132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9621965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9623805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9625639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9627469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9629283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9631213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9633155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9634984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9636807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9638658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9640509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9642317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9644164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9646003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9647859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9649680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9651533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9653370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9655235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9657179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9659517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9661382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9663254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9665079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9666932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9668753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9670609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9672411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9674235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9676086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9677917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9679749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9681699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9683660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9685498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9687331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9689157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9690997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9692841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9694668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9696482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9698327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9700170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9701994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9703814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9705717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9707657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9709469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9711309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9713128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9714968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9716766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9718603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9720417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9722262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9724069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9725910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9727725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9729647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9731561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9733392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9735226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9737145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9739870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9741821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9743710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9745560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9747405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9749243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9751133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9752955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9754903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9756879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9759115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9760989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9762823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9764642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9766493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9768337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9770150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9771986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9773815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9775674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9777483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9779458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9781442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9783295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9785125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9786986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9788829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9790702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9792776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9794680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9796527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9798405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9800244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9802084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9804039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9805967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9807797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9809633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9811496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9813345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9815188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9817021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9818882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9820728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9822579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9824422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9826277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9828225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9830150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9831983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9833833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9835723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9837555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9839418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9841267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9843142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9844972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9846836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9848679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9850537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9852429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9854359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9856214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9858111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9860494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9862377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9864224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9866103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9867923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9869757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9871616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9873463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9875295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9877256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9879234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9881087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9882931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9884776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9886633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9888481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9890327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9892165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9894007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9895844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9897683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9899508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9901421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9903368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9905183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9907036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9908869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9910734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9912641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9914410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9916256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9918127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9919957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9921819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9923731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9925628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9927541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9929397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9931243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9933105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9934955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9936795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9938656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9940500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9942334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9944166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9946029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9947879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9949795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9951719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9953571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9955422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9957315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9959509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9961400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9963259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9965099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9966950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9968797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9970685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9972513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9974528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9976501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9978375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9980215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9982087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9983920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9985769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9987591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9989449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9991287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9993157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9994985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9996824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:06.9998794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0000749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0002590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0004433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0006296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0008139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0009968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0011802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0013667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0015520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0017368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0019213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0021085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0022996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0024918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0026751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0028607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0030449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0032259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0034120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0035940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0037812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0039625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0041475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0043309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0045162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0047061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0049000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0050836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0052708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0054545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0056395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0058386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0060563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0062410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0064244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0066111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0067984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0069839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0071803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0073785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0075625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0077471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0079309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0081171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0083017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0084860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0086705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0088566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0090423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0092273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0094118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0096030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0097997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0099819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0101671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0103516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0105389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0107218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0109084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0110924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0112792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0114618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0116470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0118293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0120226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0122124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0123964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0125803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0127647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0129478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0131302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0133150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0134996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0136834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0138675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0140541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0142383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0144296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0146206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0148058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0149912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0151762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0153595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0155447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0157349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0159539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0161459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0163319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0165159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0166965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0168935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0170894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0172757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0174580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0176433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0178262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0180121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0181941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0183799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0185628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0187492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0189318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0191150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0193106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0195041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0196889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0198725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0200591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0202456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0204292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0206128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0207978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0209814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0211651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0213483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0215337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0217258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0219174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0221008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0222836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0224705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0226534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0228378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0230212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0232081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0233894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0235751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0237596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0239462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0241359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0243297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0245132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0246993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0248816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0250669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0252481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0254315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0256155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0258007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0260208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0262081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0263906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0265850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0267809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0269668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0271467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0273292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0275153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0277002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0278836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0280667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0282523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0284383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0286218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0288071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0290011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0291966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0293786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0295639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0297466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0299334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0301148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0302996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0304842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0306699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0308518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0310373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0312202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0314055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0315948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0318746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0320583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0322440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0324284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0326117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0328000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0329836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0331682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0333523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0335387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0337231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0339075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0340988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0342914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0344743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0346571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0348404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0350252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0352103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0353947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0355771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0357651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0359872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0361720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 7%] 2024-08-08T21:00:07.0363568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0365539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0367582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0369399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0371252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0373099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0374970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0376797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0378666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0380505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0382374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0384202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0386038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0387874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0389784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0391694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0393517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0395379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0397232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0399065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0400899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0402754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0404597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0406435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0408273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0410122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0411974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0413884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0415782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0417618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0419491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0421319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0423168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0425009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0426889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0428712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0430562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0432388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0434232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0436040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0437958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0439872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0441734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0443546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0445398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0447215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0449059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0450894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0452715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0454570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0456422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0458382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0460536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0462529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0464513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0466355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0468205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0470069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0471914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0473791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0475605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0477440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0479270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0481089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0482941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0484779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0486719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0488626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0490469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0492301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0494169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0495988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0497835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0499675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0501533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0503345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0505191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0507041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0508907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0510808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0512729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0514582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0516436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0518285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0520122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0521966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0523805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0525712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0527447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0529304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0531156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0532986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0534886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0536833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0538652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0540488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0542320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0544144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0546007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0547819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0549653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0551484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0553353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0555177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0557071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0559398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0561418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0563236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0565086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0566914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0568759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0570563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0572404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0574241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0576077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0577920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0579752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0581598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0583521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0585437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0587258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0589116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0590971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0592806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0594631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0596499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0598353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0600196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0602040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0603906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0605753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0607669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0609594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0611419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0613272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0615094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0616937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0618772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0620645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0622474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0624338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0626150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0628020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0629832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0631763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0633682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0635542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0637357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0639188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0641038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0642886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0644731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0646569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0648421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0650270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0652113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0653938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0655865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0657823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0660008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0661847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0663707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0665557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0667384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0669208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0671046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0672894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0674700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0676545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0678371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0680343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0682278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0684122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0685949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0687849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0689640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0691487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0693319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0695179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0696997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0698842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0700661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0702482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0704379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0706279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0708116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0709956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0711784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0713618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0715463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0717303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0719149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0720963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0722804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0724648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0726480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0728379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0730337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0732185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0733997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0735850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0737684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0739549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0741372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0743233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0745052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0746905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0748741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0750543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0752368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0754296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0756239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0758113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0760361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0762250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0764066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0765895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0767743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0769572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0771400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0773223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0775069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0776902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0778864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0780805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0782647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0784490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0786329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0788149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0789989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0791823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0793637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0795445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0797269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0799125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0800928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0802841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0804772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0806610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0808418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0810260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0812088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0813946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0815767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0817612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0819443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0821311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0823138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0824978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0826889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0828843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0830682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0832521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0834368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0836194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0838027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0839846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0841689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0843521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0845357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0847183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0849024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0850939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0852853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0854673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0856518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0858853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0860620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0862457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0864293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0866159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0867980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0869828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0871659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0873509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0875439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0877396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0879211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0881061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0882859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0884680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0886500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0888349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0890157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0891972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0893820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0895659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0897486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0899373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0901300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0903122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0904945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0906770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0908619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0910454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0912287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0914115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0915952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0917788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0919613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0921452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0923363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0925292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0927088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0928913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0930733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0932590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0934396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0936234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0938057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0939904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0941714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0943551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0945391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0947291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0949186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0950994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0952833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0954670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0956501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0958706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0960558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0962395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0964226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0966044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0967878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0969691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0971500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0973450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0975379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0977207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0979030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0980841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0982656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0984513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0986323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0988154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0989994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0991841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0993649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0995488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0997401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.0999338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1001152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1002994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1004831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1006685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1008505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1010358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1012173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1013991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1015816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1017636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1019472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1021401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1023308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1025131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1026976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1028821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1030643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1032461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1034313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1036145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1037963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1039791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1041634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1043486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1045375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1047306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1049136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1050997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1052816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1054657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1056471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1058701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1060572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1062410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1064234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1066091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1067887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1069849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1071786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1073622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1075445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1077271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1079105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1080926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1082765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1084564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1086412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1088252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1090102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1091909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1093823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1095767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1097579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1099399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1101232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1103055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1104849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1106683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1108504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1110347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1112160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1113995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1115813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1117734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1119615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1121436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1123257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1125116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1126930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1128767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1130603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1132468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1134280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1136103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1137964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1139799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1141626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1143587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1145508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1147328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1149149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1150969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1152806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1154642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1156468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1158688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1160570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1162416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1164222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1166052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1169767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1173427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1176954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1180455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1184079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1187580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1191055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1194582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1198097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1201641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1205147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1208670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1212299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1215813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1219418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1222921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1226403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1229941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1233447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1236942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 8%] 2024-08-08T21:00:07.1249766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1253340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1256916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1260915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1264435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1267959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1271674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1275329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1278847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1282389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1285919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1289438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1293034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1296601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1300127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1303631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1307145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1310675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1314143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1317716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1321315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1324840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1328354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1331861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1335355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1338881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1342386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1345897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1349405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1352937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1356725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1360762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1364476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1368141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1371674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1375174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1378693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1382221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1385752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1389265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1392784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1396299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1399801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1403296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1406801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1410395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1414025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1417512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1421019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1424547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1428069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1431550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1435063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1438590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1442090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1445584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1449104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1452612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1456107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1460249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1463926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1467462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1470976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1474488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1477973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1481480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1484981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1488449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1491947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1495467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1498951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1502433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1506036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1509940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1513456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1516949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1520472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1523984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1527493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1530993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1534572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1538061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1541592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1545111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1548609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1552253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1555861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1560010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1563591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1567118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1570599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1574084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1577595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1581132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1584629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1588123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1591643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1595149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1598867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1602486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1606006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1609554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1613102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1616652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1620183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1623756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1627317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1630857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1634414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1637995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1641533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1645158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1648786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1652325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1655847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1661476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1665127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1668674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1672224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1675762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1679304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1682874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1686429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1689950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1693665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1697347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1700883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1704410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1708223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1711832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1715366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1718916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1722473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1726037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1729597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1733146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1736678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1740305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1743916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1747417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1750944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1754508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1758105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1763381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1766931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1770501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1774034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1777557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1781095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1784633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1788336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1791962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1795514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1799067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1802617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1806155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1809686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1813235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1816779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1820301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1823827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1827371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1830878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1834466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1838060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1841594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1845116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1848639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1852188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1855730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1859720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1863262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1866801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1870362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1873917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1877443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1880979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1884709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1888422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1891959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1895512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1899095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1902642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1906181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1909711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1913250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1916775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1920297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1923831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1927416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1931875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1935555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1939100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1942659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1946201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1949811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1953260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1956813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1960911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1964460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1968021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1971602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1975158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1978854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1982527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1986093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1989655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1993203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.1996745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2000296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2003825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2007321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2010860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2014432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2017983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2021508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2025141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2028792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2032314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2035897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2039381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2042933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2046465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2049994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2053519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2057127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2061090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2064628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2068175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2071882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2075551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2079056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2082590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2086133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2089649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2093146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2096674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2100217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2103722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2107238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2110776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2114317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2117927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2121549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2125092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2128628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2132181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2135713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2139270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2142857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2146406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2149951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2153482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2157094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2161042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2164735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2168406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2171949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2175465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2178994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2182526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware)[32m [ 9%] 2024-08-08T21:00:07.2186098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2189653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2193169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2196702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2200258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2203824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2207337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2210995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2214609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2218135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2221648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2225194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2228750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2232296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2235846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2239374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2242929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2246465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2250008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2253525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2257205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2261312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2264823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2268359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2271902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2275430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2278943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2282482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2286045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2289572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2293070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2296607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2300141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2303855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2307499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2311051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2314614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2318152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2321685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2325201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2328764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2332314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2335874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2339381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2342916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2346426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2350018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2353637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2357225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2361170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2364682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2368214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2371746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2375277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2378802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2382323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2385882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2389397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2392913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2396612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2400273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2403812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2407340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2410895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2414444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2417991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2421531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2425069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2428608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2432137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2435633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2439149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2442780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2446403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2449914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2453444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2457002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2460995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2464520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2468051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2471603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2475118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2478630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2482173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2485721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2489413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2493055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2496583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2500122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2503651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2507165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2510684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2514223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2517710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2521201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2524717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2528254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2531752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2535254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2538857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2542480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2545990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2549490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2553022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2556552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2560553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2564095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2567631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2571197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2574753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2578258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2581798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2585483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2589156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2592680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2596213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2599754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2603250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2606760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2610299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2613851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2617378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2620911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2624436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2627959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2631569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2635185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2638716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2642265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2645803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2649332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2652854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2656423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2660461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2664020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2667571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2671130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2674641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2678302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2681948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2685477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2689003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2692515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2696038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2699559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2703087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2706608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2710139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2713684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2717214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2720720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2724320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2727954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2731482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2734998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2738547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2742100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2745620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2749149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2752672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2756214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2760146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2763712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2767233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2770887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2774529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2778029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2781564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2785118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2788640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2792154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2795675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2799211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2802736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2806257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2809783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2813318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2816903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2820526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2824053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2827614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2831152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2834662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2838185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2841709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2845250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2848763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2852293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2855826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2859744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2863430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2867051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2870584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 9%] 2024-08-08T21:00:07.2874100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2877622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2881135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2884663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2888169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2891671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2895190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2898736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2902274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2905786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2909416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2913061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2916588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2920368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2923926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2927483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2931016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2934550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2938065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2941598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2945105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2948636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2952136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2955783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2959812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2963358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2966876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2968745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2970565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2972414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2974242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2976101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2977913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2979741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2981604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2983452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2985444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2987397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2989245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2991087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2992932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2994769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2996599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.2998439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3000263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3002068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3003915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3005761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3007584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3009483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3011467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3013316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3015141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3016955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3018780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3020628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3022438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3024295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3026124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3027987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3029808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3031641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3033478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3035397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3037293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3039130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3040970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3042774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3044589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3046395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3048239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3050077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3051907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3053731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3055566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3057430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3059760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3061729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3063576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3065442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3067261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3069085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3070927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3072820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3074622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3076467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3078300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3080170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3081988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3083933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3085837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3087686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3089498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3091339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3093179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3095020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3096838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3098680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3100506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3102345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3104181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3105997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3107915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3109832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3111676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3113484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3115342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3117185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3119018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3120848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3122689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3124548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3126368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3128200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3130036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3131938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3133822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3135644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3137472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3139329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3141131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3142962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3144790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3146651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3148460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3150301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3152132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3153982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3155867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3157843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3160053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3161942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3163764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3165591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3167432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3169269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3171109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3172929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3174761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3176595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3178404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3180339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3182295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3184122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3185938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3187769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3189613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3191545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3193263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3195082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3196900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3198754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3200568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3202406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3204317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3206257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3208063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3209896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3211728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3213586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3215396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3217237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3219057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3220890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3222685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3224511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3226326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3228262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3230138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3231956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3233790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3235624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3237453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3239257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3241099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3242934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3244750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3246559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3248403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3250234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3252050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3253967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3255866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3257752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3259926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3261795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3263605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3265445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3267246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3269066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3270880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3272731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3274528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3276348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3278296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3280259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3282056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3283888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3285717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3287556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3289376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3291207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3293057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3294891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3296722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3298549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3300392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3302310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3304231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3306056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3307891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3309722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3311540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3313348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3315191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3317026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3318826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3320665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3322496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3324338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3326217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3328133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3329957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3331808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3333624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3335457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3337288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3339145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3340971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3342811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3344639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3346522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3348309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3350224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3352141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3353964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3355786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3357643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3359836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3361693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3363526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3365351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3367201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3369022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3370849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3372661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3374621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3376569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3378386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3380198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3382034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3383897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3385732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3387549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3389387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3391237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3393050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3394902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3396716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3398682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3400563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3402383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3404197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3406048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3407863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3409679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3411521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3413360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3415180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3416997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3418834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3420662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3422475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3424359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3426283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3428111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3429932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3431748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3433587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3435415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3437263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3439053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3440865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3442703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3444490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3446308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3448215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3450144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3451939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3453764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3455587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3457425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3459634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3461540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3463364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3465229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3467020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3468868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3470697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3472666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3474615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3476435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3478286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3480153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3481964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3483797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3485645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3487468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3489305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3491094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3492940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3494770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3496666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3498574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3500414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3502249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3504058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3505886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3507701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3509548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3511363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3513197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3515063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3516903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3518712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3520627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3522552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3524408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3526219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3528069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3529883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3531719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3533520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3535342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3537178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3539013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3540843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3542667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3544586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3546498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3548325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3550143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3551987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3553814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3555632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3557438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3559691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3561553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3563370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3565210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3567025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3569000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3570916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3572744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3574557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3576399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3578196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3580019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3581833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3583678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3585473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3587297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3589115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3590953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3592746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3594628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3596554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3598379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3600205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3602029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3603873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3605714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3607550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3609367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3611212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3613056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3614882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3616697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3618608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3620531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3622345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3624158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3625987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3627833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3629633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3631476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3633294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3635142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3636957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3638785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3640596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3642522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3645155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3646976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3648801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3650659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3652468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3654302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3656155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3658012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3660171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3662013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3663852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3665671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3667615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3669550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3671380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3673209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3675036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3676839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3678670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3680500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3682326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3684124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3685964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3687798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3689599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3691508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3693419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3695269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3697082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3698923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3700744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3702584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3704393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3706233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3708043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3709880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3711681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3713501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3715426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3717327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3719143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3720948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3722781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3724618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3726437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3728242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3730089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3731914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3733740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3735554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3737413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3739303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3741242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3743034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3744868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3746700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3748511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3750343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3752159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3753999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3755797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 10%] 2024-08-08T21:00:07.3757663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3759839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3761751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3763553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3765507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3767436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3769283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3771081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3772913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3774724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3776548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3778369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3780169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3782011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3783851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3785663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3787473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3789434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3791347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3793158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3794973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3796798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3798611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3800418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3802213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3804034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3805850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3807647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3809474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3811280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3813206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3815083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3816900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3818712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3820565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3822360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3824180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3826004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3827860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3829661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3831493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3833325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3835168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3837049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3838948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3840773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3842586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3844402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3846209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3848033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3849863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3851702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3853480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3855313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3857146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3859370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3861342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3863326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3865183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3867008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3868870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3870732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3872614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3874456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3876326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3878168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3880054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3881903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3883762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3885686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3887657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3889478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3891327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3893180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3895066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3896894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3898754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3900614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3902474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3904333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3906180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3908040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3909977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3911921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3913759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3915632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3917508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3919368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3921212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3923090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3924957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3926810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3928662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3930528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3932377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3934277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3936220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3938061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3939947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3941787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3943662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3945492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3947373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3949218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3951078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3952936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3954800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3956620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3959024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3961041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3962917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3964753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3966621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3968460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3970317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3972175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3974172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3976042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3977894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3979735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3981562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3983504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3985453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3987293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3989134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3990996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3992844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3994681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3996513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.3998387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4000245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4002079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4003944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4005798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4007752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4009673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4011534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4013391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4015284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4017119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4018981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4020824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4022701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4024518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4026371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4028219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4030093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4032001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4033921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4035789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4037640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4039501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4041335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4043192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4045058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4046912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4048749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4050621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4052490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4054344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4056262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4058253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4060531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4062406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4064257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4066144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4067974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4069801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4071645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4073487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4075362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4077194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4079056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4081045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4083039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4084893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4086735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4088579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4090449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4092276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4094137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4095979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4097847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4099686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4101534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4103397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4105331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4107261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4109094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4110942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4112791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4114622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4116454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4118314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4120174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4122030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4123847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4125732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4127558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4129474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4131390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4133231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4135111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4136944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4138815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4140654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4142543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4144380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4146238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4148096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4149988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4151820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4153755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4155737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4157603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4159813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4161702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4163551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4165433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4167270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4169111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4170974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4172858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4174696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4176536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4178535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4180524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4182357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4184222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4186080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4187938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4189794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4191644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4193506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4195360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4197214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4199062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4200895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4202831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4204731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4206573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4208412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4210288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4212113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4213990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4215842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4217709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4219514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4221370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4223217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4225084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4226990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4228930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4230792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4232641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4234502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4236340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4238207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4240075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4241921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4243762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4245625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4247479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4249313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4251216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4253202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4255018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4256865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4259098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4260991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4262871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4264677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4266526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4268361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4270231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4272072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4273950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4275899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4277903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4279732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4281597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4283447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4285321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4287169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4289014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4290842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4292695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4294512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4296370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4298207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4300133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4302063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4303893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4305753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4307608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4309448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4311281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4313138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4314980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4316819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4318657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4320520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4322370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4324300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4326201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4328076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4329906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4331736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4333583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4335412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4337264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4339073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4340930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4342746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4344620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4346437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4348363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4350286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4352144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4353969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4355817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4357659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4359918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4361827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4363668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4365530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4367390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4369246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4371085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4373061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4375041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4376891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4378739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4380601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4382442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4384281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4386113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4387973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4389833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4391672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4393517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4395379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4397293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4399201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4401054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4402898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4404800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4406607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4408462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4410320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4412200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4414034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4415901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4417754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4419629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4421538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4423487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4425323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4427169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4429011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4430836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4432690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4434546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4436389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4438225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4440084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4441939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4443795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4445705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4447661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4449497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4451382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4453197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4455073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4456923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4459170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4461042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4462885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4464763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4466622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4468464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4470426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4472402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4474218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4476063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4477908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4479775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4481838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4483735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4485571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4487436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4489262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4491189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4492959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4494894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4496820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4498651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4500511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4502368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4504213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4506052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4507912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4509756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4511592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4513432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4515282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4517116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4519014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4520919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4522767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4524613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4526437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4528288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4530123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4531986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4533799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4535634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4537478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4539376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4541190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4543136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4545128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4547008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4548844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4550709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4552551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4554435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4556277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4558160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4560377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4562238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4564083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4565904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4567896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4569865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4571712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4573554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4575421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4577268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4579112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4580957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4582815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4584662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4586511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4588357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4590200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4592156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4594080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4595934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4597782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4599666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4601490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4603344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4605185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 11%] 2024-08-08T21:00:07.4607054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4608866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4610716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4612557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4614418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4616313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4618270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4620109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4621977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4623802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4625633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4627489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4629344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4631183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4633013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4634871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4636728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4638568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4640480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4642417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4644268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4646115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4647956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4649801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4651634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4653468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4655287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4657119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4659393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4661296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4663142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4665106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4667086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4668900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4670756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4672606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4674480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4676306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4678163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4680027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4681900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4683740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4685604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4687446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4689378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4691312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4693161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4695008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4696865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4698701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4700522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4702382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4704246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4706085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4707924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4709796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4711645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4713562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4715497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4717348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4719193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4721023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4722877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4724717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4726591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4728424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4730276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4732118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4734001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4735831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4737770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4739677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4741612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4743345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4745192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4747023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4748887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4750710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4752548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4754399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4756250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4758140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4760794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4762971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4764951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4766788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4768625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4770487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4772335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4774174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4776008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4777872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4779718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4781565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4783407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4785230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4787162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4789059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4790894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4792726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4794593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4796413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4798261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4800102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4801999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4803815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4805667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4807507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4809363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4811263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4813203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4815040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4816916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4818744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4820574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4822426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4824276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4826128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4827960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4829804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4831649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4833480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4835378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4837328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4839174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4841016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4842850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4844698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4846537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4848375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4850213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4852093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4853962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4855781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4857616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4859992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4862060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4863876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4865736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4867568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4869436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4871249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4873106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4874931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4876768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4892755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4894619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4896511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4898499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4900430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4902264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4904122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4905975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4907816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4909640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4911499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4913355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4915276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4917176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4919060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4920923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4922839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4924787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4926630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4933385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4935312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4937170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4939004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4940852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4942673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4944515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4946352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4948211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4950019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4952015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4953941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4955781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4957584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4959914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4961748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4963612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4965438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4967282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4969140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4971022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4972871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4974740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4976737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4978747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4980601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4982451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4984316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4986175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4988019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4989859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4991734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4993592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4995449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4997298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.4999164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5001107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5003046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5004889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5006750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5008613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5010455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5012310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5014177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5016065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5017908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5019777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5021636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5023525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5025441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5027391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5029235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5031102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5032935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5034796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5036642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5038527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5040366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5042225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5044072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5045940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5047786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5049711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5051659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5053561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5055410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5057259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5059516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5061404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5063267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5065125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5066989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5068849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5070706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5072545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5074534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5076490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5078309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5080153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5081996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5083871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5085693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5087547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5089395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5091271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5093102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5094960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5096809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5098758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5100668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5102523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5104382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5106264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5108111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5109953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5111823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5113696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5115557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5117407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5119269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5121113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5123026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5124940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5126800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5128667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5130519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5132358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5134213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5136072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5137935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5139776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5141646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5143570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5145396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5147325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5149270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5151148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5153031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5154913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5156758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5158985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5160868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5162748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5164586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5166457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5168288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5170138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5172119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5174114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5175944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5177790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5179661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5181521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5183372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5185220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5187090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5188942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5190792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5192639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5194504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5196449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5198387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5200226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5202088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5203956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5205812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5207653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5209493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5211365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5213181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5215027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5216875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5218745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5220647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5222586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5224424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5226296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5228127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5229979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5231825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5233755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5235605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5237466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5239327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5241220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5243057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5244992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5246966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5248834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5250696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5252585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5254440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5256290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5258143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5260356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5262235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5264097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5265948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5267784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5269767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5271746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5273592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5275440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5277284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5279153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5280976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5282833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5284686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5286555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5288390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5290254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5292092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5294040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5295959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5297815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5299651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5301513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5303325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5305164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5307000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5308860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5310704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5312542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5314403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5316255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5318857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5320812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5322667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5324520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5326370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5328203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5330060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5331921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5333780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5335623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5337485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5339347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5341195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5343116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5345054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5346912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5348731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5350580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5352462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5354337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5356169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5358033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5360210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5362108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5363943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5365792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5367770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5369757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5371581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5373444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5375302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5377162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5379012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5380855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5382723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5384584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5386444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5388295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5390150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5392067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5393998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5395823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5397685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5399544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5401400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5403234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5405093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5406948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5408769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5410617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5412462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5414332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5416235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5418178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5420020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5421897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5423730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5425586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5427419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5429290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5431131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5432983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5434808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5436676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5438488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5440398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5442349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5444191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5446031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5447875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5449729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5451576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5453458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5455297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5457153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5459400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5461285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5463119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5465113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5467080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5468930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5470774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5472626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5474501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 12%] 2024-08-08T21:00:07.5476328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5478191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5480033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5481897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5483716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5485565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5487438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5489347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5491265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5493116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5494961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5496830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5498650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5500496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5502342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5504221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5506046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5507885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5509757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5511614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5513542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5515463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5517333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5519194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5521050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5522886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5524740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5526583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5528423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5530246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5532104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5534013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5535859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5537817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5539757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5541624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5543457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5545311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5547148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5549012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5550842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5552746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5554596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5556479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5558387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5560549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5562525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5564525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5566355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5568231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5570074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5571918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5573755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5575593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5577459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5579309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5581165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5583011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5584874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5586812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5588743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5590577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5592430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5594293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5596135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5597967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5599831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5601698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5603522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5605379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5607230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5609101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5611007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5612956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5614785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5616645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5618470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5620319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5622159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5624030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5625860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5627709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5629548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5631424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5633244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5635171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5637094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5638947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5640797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5642646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5644519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5646381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5648242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5650092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5651996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5653869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5655731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5657575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5659948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5661938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5663774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5665602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5667467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5669315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5671142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5672999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5674849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5676711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5678535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5680391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5682227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5684223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5686136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5687992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5689843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5691726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5693558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5695418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5697274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5699160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5700990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5702831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5704691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5706530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5708449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5710378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5712232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5714084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5715941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5717780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5719638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5721489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5723339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5725170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5727023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5728875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5730711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5732614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5734537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5736408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5738242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5740109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5741947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5743806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5745629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5747489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5749322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5751179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5753042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5754882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5756791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5759089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5760942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5762796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5764636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5766477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5768310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5770139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5772005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5773857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5775711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5777556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5779424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5781407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5783370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5785206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5787074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5788938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5790785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5792620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5794475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5796325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5798159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5799991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5801842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5803706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5805629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5807569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5809405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5811285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5813119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5814968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5816804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5818673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5820504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5822355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5824201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5826088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5827909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5829851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5831770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5833681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5835538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5837387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5839238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5841076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5842911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5844746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5846601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5848449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5850298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5852167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5854108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5856041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5857879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5860071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5861960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5863801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5865622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5867475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5869324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5871192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5873016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5874880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5876716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5878710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5880650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5882501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5884322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5886178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5887996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5889837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5891676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5893545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5895361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5897195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5899052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5900896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5902824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5904741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5906588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5908431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5910271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5912109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5913962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5915824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5917674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5919509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5921373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5923227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5925074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5926982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5928901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5930750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5932554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5934395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5936237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5938104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5939925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5941786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5943616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5945477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5947302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5949147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5951082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5953025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5954845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5956687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5958857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5960750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5962582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5964412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5966266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5968112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5969956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5971793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5973626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5975590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5977545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5979362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5981208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5983056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5984898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5986723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5988576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5990425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5992239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5994082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5995932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5997792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.5999607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6001537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6003469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6005339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6007175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6009036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6010881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6012754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6014581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6016441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6018273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6020144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6021955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6023774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6025711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6027642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6029473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6031305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6033228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6035018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6036857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6038695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6040543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6042457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6044286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6046061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6047911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6049836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6051757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6053622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6055462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6057335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6059527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6061415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6063247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6065092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6066900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6068741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6070569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6072415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6074402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6076372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6078197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6080054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6081873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6083712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6085541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6087385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6089210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6091036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6092897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6094753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6096590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6098490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6100427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6102269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6104112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6105947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6107780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6109614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6111441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6113251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6115094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6116932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6118748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6120585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6122496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6124435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6126246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6128093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6129927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6131782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6133629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6135486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6137315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6139169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6140986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6142832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6144659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6146590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6148492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6150310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6152180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6154016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6155827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6157633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6159839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6161709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6163533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6165351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6167195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6169024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6170986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6172916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6174765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6176600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6178438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6180259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6182088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6183955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6185777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6187619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6189450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6191318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6193157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6196861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6200469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6204007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6207527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6211100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6214573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6218129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6221668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6225299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6228732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6232381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6235915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6239400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6243011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6246635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6250173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6253740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6257286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6261246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6264782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6268311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6271858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6275404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6278939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6282465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6285971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6289642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6293272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6296771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6300291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6303837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6307373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6310870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6314394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6317937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6321460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6324965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6328496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6332030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6335629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6339222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6342749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6346303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6349828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6353401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6356923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6360865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6364407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6367917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6371430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6374957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6378452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6381931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6385570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6389220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6392734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6396247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6399770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6403287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6406800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6410306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6413824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6417368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6420916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6424436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6427962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6431592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6435275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6438797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 13%] 2024-08-08T21:00:07.6442330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6445897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6449433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6452982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6456514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6460399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6463932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6467438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6470961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6474494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6478130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6481776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6485301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6488847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6492377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6495882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6499389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6502927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6506438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6509939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6513467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6517025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6520550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6524140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6527753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6531289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6534941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6538473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6541987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6545508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6548997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6552529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6556042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6559926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6563475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6566972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6570606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6574261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6577772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6581275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6584802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6588329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6591846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6595342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6598879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6602422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6605956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6609481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6613001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6616622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6620215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6623728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6627250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6630777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6634276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6637764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6641269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6644808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6648307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6651821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6655398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6659289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6662969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6666626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6670142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6673670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6677199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6680699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6684224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6687779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6691293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6694803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6698399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6701885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6705406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6709007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6712592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6716105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6719602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6723093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6726616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6730144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6733754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6737239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6740756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6744286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6747827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6751319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6754954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6758873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6762418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6765913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6769432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6772983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6776495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6779994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6783505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6787030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6790541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6794050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6797559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6801061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6804683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6808270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6811770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6815297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6818820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6822314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6825835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6829348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6832852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6836346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6839869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6843408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6846930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6850540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6854948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6858876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6862460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6865990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6869523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6873069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6876617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6880107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6883624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6887145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6890652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6894137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6897782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6901413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6904914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6908414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6911935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6915464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6918984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6922497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6926020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6929587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6933145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6936692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6940248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6943897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6947553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6951073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6954681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6958268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6962200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6965749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6969306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6972843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6976373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6979890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6983429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6986998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6990675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6994323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.6997869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7001447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7005002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7008526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7012077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7015646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7019201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7022716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7026278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7029873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7033481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7037145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7040793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7044374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7047921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7051474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7055064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7058975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7062561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7066081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7069607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7073173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7076736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7080273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7083948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7087629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7091167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7094697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7098251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7101794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7105344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7108877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7112423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7115982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7119524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7123069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7126611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7130264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7133899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7137433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7140981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7144519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7148037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7151551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7155145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7159042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7162618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7166132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7169682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7173231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7176902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7180558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7184095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7187672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7191220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7194752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7198309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7201895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7205474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7209016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7212639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7216155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7219723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7223347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7226987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7230541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7234081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7237615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7241132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7244707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7248256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7251791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7255382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7259330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7262902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7266432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7270106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7273789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7277330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7280865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7284424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7287982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7291544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7295099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7298648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7302227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7305795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7309346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7312881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7316524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7320141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7323662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7327198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7330771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7334368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7337891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7341451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7345016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7348549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7352080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7355668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7359550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7363256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7366926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7370475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7374045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7377604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7381139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7384677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7388248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7391805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7395335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7398882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7402438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7405961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7409531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7413208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7416767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7420321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7423854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7427397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7430933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7434473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7438007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7441558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7445128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7448708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7452244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7455915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7459953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7463539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7467094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7470656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7474234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7477796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7481332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7484894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7488443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7491982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7495500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7499047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7502741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7506443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7509927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7513476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7517045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7520588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7524115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7527644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7531210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7534745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7538286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7541840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7545398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7549042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7552690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7556249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7560170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7563761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7567291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7570821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7574367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7577879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7581387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7584929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7588502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7592035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7595683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7599341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7602885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7606426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7609953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7613498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7617055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7620595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7624113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7627652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7631212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7634826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7638381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7642028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7645680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7649220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7652808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7656364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7660251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7663795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7667312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7670851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7674383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7677916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7681446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7684979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7688659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7692300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7695794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7699335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7702894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7706429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7709944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7713492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7717064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7720609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7724154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7727714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7731275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7734923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7738544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7742077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7745608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7749133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7752678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7756212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7760116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7763688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7767213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7770731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7774298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7777831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7781488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7785145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7788694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7792210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7795733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7799282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7802840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7806374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7809923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7813462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7817009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7820560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7824081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7827690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7831303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7834815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7838308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7841820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7845362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7848878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7852398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7855978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7859873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7863408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7866926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7870460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7874133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7877808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7881327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7884870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7888429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7891983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7895510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7899060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7902625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7906169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7909704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7913228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7916759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7920389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7924011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7927545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7931097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7934687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7938207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7941737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7945283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7948825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7955435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7961862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7965452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7968986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7972675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7976315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7979889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7983452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7987001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7988852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7990725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7992560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7994423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7996245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.7998239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8000142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8001983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8003820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8005738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8007605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8009454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8011289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8013154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8014974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8016811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8018663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8020505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8022412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8024317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8026177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 14%] 2024-08-08T21:00:07.8028035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8029948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8031835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8033698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8035554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8037401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8039244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8041093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8042927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8044754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8046629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8048517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8050378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8052204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8054159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8056046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8057907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8060367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8062256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8064085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8065952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8067771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8069608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8071549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8073553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8075363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8077210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8079105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8081013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8082854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8084690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8086530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8088360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8090189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8092013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8093857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8095744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8097633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8099450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8101291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8103170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8105054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8106872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8108730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8110576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8112392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8114246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8116097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8117967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8119833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8121752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8123598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8125472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8127342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8129243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8131070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8132931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8134745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8136582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8138416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8140284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8142100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8143972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8145878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8147722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8149555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8151421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8153360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8155212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8157058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8159263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8161152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8163009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8164861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8166696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8168645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8170577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8172430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8174272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8176154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8178090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8179903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8181754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8183591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8185449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8187273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8189131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8190962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8192873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8194744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8196600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8198435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8200339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8202199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8204046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8205888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8207761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8209572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8211405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8213260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8215099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8217002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8218888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8220739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8222571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8224439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8226302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8228150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8229996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8231838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8233722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8235581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8237431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8239250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8241146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8243043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8244910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8246740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8248638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8250526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8252396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8254264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8256129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8257963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8260196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8262064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8263923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8265841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8267778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8269592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8271436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8273340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8275246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8277081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8278917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8280768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8282603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8284433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8286267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8288113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8289994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8291877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8293708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8295562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8297450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8299350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8301178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8303029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8304881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8306697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8308542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8310365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8312216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8314066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8315954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8317787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8319643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8321493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8323384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8325209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8327065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8328880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8330710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8332535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8334395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8336217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8338087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8339999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8341849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8343679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8345559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8347458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8349298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8351137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8353014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8354857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8356688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8358850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8360716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8362656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8364574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8366390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8368230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8370119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8372046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8373864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8386588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8388589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8390468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8392296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8394155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8395998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8397989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8399891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8401745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8403579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8405479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8407375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8409213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8411057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8412905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8414735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8416554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8418407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8420246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8422070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8423940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8425852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8427680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8429515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8431381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8433267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8435101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8436914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8438761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8440593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8442460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8444280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8446120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8447987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8449891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8451779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8453610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8455483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8457373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8459611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8461488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8463317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8465182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8466995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8468820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8470655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8472597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8474501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8476318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8478176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8480092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8481989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8483819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8485682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8487533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8489441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8491201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8493057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8494924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8496776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8498659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8500487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8502339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8504189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8506068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8507900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8509759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8511573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8513409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8515244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8517109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8518915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8520797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8522677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8524537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8526351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8528227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8530122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8531970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8533865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8535705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8537557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8539406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8541259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8543093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8544976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8546860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8548689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8550505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8552397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8554325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8556162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8557985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8560291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8562178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8563994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8565836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8567673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8569625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8571534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8573384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8575225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8577154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8579037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8580888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8582719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8584589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8586401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8588250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8590079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8591914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8593791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8595663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8597513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8599354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8601232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8603099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8604941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8606799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8608637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8610451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8612290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8614133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8615966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8617829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8619732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8621571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8623387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8625271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8627147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8629001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8630818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8632673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8634496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8636340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8638151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8639982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8641848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8644567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8646388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8648218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8650087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8651996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8653847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8655672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8657527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8659782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8661655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8663488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8665349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8667301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8669207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8671042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8672894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8674795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8676702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8678530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8680372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8682209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8684041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8685860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8687689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8689543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8691353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8693239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8695118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8696966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8698778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8700660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8702535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8704395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8706214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8708060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8709904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8711777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8713592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8715436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8717316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8719212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8721049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8722883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8724760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8726635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8728464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8730284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8732135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8733964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8735800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8737620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8739463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8741346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8743224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8745039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8746883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8748762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8750621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8752458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8754339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8756200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8758008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8760202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8762065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8763922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8765817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8767739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8769561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8771408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8773269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8775185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8777007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8778868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8780684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8782502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8784344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8786184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8788013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8789940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8791845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8793688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8795515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8797394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8799297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8801150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8802994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8804826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8806671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8808517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8810366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8812201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8814069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8815966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8817783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8819624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8821497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8823403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8825212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8827060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8828901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8830754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8832568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8834473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8836312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8838213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8840076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8841932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8843763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8845672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8847558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8849378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8851224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8853108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8854939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8856774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8859036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8860905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8862824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8864710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8866559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8868387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8870269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8872158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8873993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8875831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8877650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8879485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8881307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8883162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8884966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 15%] 2024-08-08T21:00:07.8886848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8888733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8890596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8892404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8894284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8896164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8898021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8899833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8901682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8903500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8905328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8907157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8908969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8910851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8912735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8914558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8916376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8918218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8920101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8921975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8923783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8925625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8927454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8929276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8931107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8932963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8934798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8936653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8938562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8940386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8942243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8944100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8945998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8947819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8949672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8951471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8953350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8955182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8957039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8959233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8961274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8963180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8965029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8966839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8968728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8970632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8972467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8974295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8976112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8977950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8979788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8981620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8983433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8985317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8987202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8989037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8990860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8992737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8994609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8996450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.8998240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9000066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9001910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9003722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9005560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9007378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9009276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9011128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9012969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9014797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9016704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9018560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9020405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9022247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9024114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9025936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9027757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9029606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9031448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9033341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9035223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9037067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9038897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9040765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9042624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9044466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9046304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9048135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9049948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9051788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9053681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9055529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9057398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9059685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9061600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9063430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9065410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9067334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9069210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9071046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9072920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9074774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9076654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9078492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9080348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9082246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9084179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9086007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9087854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9089733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9091629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9093468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9095308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9097168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9099025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9100873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9102714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9104577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9106473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9108370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9110213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9112076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9113979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9115900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9117737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9119601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9121464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9123316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9125157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9126995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9128853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9130713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9132602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9134513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9136395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9138274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9140187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9142020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9143888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9145717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9147570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9149409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9151287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9153152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9155054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9156947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9159177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9161088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9163004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9164999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9166856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9168703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9170552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9172413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9174250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9176096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9177924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9179873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9181801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9183643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9185473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9187370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9189271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9191088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9192938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9194790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9196662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9198491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9200364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9202217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9204140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9206029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9207898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9209740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9211681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9213561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9215433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9217259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9219128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9220944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9222777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9224640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9226500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9228393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9230280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9232145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9233997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9235876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9237763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9239618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9241477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9243329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9245168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9247034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9248903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9250764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9252651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9254615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9256485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9258385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9260719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9262643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9264503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9266323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9268176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9270055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9271889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9273718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9275573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9277472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9279421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9281248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9283098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9284974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9286885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9288703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9290540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9292405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9294259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9296102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9297943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9299801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9301695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9303607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9305447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9307298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9309174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9311055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9312872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9314727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9316578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9318424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9320255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9322091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9323951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9325810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9327701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9329549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9331427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9333301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9335217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9337061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9338939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9340778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9342634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9344480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9346360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9348195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9350115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9351998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9353902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9355742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9357620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9359939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9361820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9363677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9365524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9367386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9369240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9371087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9372925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9374893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9376826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9378669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9380497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9382436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9384375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9386195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9388052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9389896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9391763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9393596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9395470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9397299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9399216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9401081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9402930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9404764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9406675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9408548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9410395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9412224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9414095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9415911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9417742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9419601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9421452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9423331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9425218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9427086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9428933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9430826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9432712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9434634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9436498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9438356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9440196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9442042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9443888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9445721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9447548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9449454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9451354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9453209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9455061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9456952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9459229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9461084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9462942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9464787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9466656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9468486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9470359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9472182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9474158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9476059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9477911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9479749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9481682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9483629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9485433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9487283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9489119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9491003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9492778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9494637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9496473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9498362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9500245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9502169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9503934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9505806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9507684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9509529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9511373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9513207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9515035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9516880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9518743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9520551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9522446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9524337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9526195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9528014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9529942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9531820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9533667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9535477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9537317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9539142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9541003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9542816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9544653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9546525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9548423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9550258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9552078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9554071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9555971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9557817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9560023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9561920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9563779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9565627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9567474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9569330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9571324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9573274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9575106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9576963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9578883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9580755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9582596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9584440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9586306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9588123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9589987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9591832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9593691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9595556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9597455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9599296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9601164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9603042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9604938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9606778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9608654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9610483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9612326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9614185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9616049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9617889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9619772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9621661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9623501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9625338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9627211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9629112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9630952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9632795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9634626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9636472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9638320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9640164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9641989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9643869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9645787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9647614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9649465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9651351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9653315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9655150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9657011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9659233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9661135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9662961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9664819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9666646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9668585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9670489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9672334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9674167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9676079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9677983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9679816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9681667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9683513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9685342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9687170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9689026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9690857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9692737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9694616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9696479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9698323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9700205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9702081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9703931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9705784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9707615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9709458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9711277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9713131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9714937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9716814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9718699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9720560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9722370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9724257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9726133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9727990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9729803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9731659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 16%] 2024-08-08T21:00:07.9733487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9735426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9737252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9739086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9741006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9742911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9744760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9746602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9748503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9750402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9752247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9754123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9755984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9757816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9760033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9761890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9763747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9765718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9767643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9769484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9771320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9773251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9775131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9776976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9778811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9780684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9782510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9784364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9786211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9788092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9789962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9791875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9793712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9795585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9797456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9799356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9801179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9803019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9804860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9806675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9808532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9810384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9812230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9814102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9816010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9817860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9819699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9821578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9823471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9825302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9827136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9828972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9830824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9832667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9834496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9836347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9838222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9840134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9841960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9843806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9845669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9847602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9849417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9851255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9853138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9855008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9856824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9859074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9860955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9862914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9864808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9866634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9868487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9870394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9873078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9874913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9876772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9878636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9880482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9882322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9884186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9886038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9887929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9889821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9891669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9893500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9895364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9897237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9899071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9900932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9902751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9904604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9906437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9908302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9910119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9912004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9913883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9915744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9917553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9919438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9921328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9923189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9925008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9926864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9928696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9930541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9932386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9934218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9936052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9937928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9939851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9941667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9943516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9945409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9947289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9949109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9950965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9952802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9954678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9956505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9958648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9960578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9962516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9964453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9966298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9968168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9970053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9971977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9973813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9975687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9977513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9979367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9981196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9983055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9984860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9986736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9988614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9990473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9992293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9994163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9996061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9997908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:07.9999743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0001567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0003415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0005263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0007113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0008926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0010825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0012719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0014561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0016403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0018297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0020184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0022030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0023872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0025697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0027542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0029359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0031199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0033030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0034985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0036878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0038725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0040565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0042485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0044347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0046186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0048014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0049865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0051674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0053555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0055404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0057241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0059544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0061479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0063330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0065170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0067072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0068962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0070794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0072619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0074439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0076245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0078086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0079919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0081743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0083601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0085497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0087333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0089137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0091014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0092889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0094745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0096562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0098415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0100249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0102109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0103932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0105784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0107656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0109569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0111391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0113244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0115105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0117002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0118805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0120623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0122471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0124311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0126140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0127965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0129812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0131725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0133611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0135437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0137395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0139192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0141092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0142930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0144800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0146663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0148522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0150361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0152215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0154151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0156029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0157944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0160123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0162017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0163923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0165839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0167674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0169548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0171389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0173246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0175083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0176959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0178786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0180697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0182619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0184485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0186328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0188212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0190130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0191985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0193845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0195692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0197558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0199419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0201281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0203120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0205017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0206911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0208749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0210577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0212472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0214389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0216236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0218079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0219925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0221794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0223622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0225485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0227326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0229237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0231117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0232972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0234808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0236722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0238595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0240455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0242294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0244172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0245999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0247864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0249700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0251536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0253457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0255337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0257199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0259426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0261386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0263292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0265137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0266980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0268819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0270650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0272507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0274365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0276210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0278116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0280052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0281917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0283747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0285648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0287550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0289426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0291257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0293117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0294951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0296820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0298644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0300493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0302367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0304285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0306113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0307966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0309871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0311791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0313613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0315447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0317308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0319198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0321073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0322972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0324841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0326804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0328711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0330554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0332414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0334379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0336293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0338134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0339988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0341838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0343673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0345500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0347340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0349207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0351107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0353017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0354896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0356766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0359094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0361017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0362850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0364721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0366553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0368410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0370251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0372130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0373958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0375883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0377800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0379670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0381491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0383377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0385275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0387113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0388947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0390782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0392643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0394489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0396326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0398155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0400048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0401959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0403798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0405627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0407536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0409442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0411287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0413128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0414984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0416862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0418697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0420560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0422408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0424331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0426215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0428080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0429907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0431808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0433679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0435529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0437366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0439241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0441072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0442927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0444772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0446623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0448509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0450397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0452262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0454151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0456034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0457922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0460139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0462027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0463882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0465723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0467582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0469434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0471279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0473204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0475120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0476959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0478770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0480679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0482663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0484518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0486253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0488103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0490014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0491798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0493625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0495475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0497367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0499286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0501104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0502946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0504830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0506754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0508610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0510420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0512281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0514131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0515971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0517811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0519659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0521542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0523422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0525246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0527098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0528984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0530868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0532689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0534536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0536382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0538217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0540048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0541887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0543753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0545567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0547461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0549353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0551215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0553043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0554983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0556873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0559088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0560955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0562811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0564637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0566496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0568307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0570155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0572071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0573996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0575822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0577655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0579575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 17%] 2024-08-08T21:00:08.0581516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0583363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0585200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0587057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0588899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0590733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0592563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0594409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0596310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0598203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0600025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0601878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0603770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0605635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0607483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0609308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0611152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0612957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0614802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0616641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0618489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0620348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0622249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0624074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0625927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0627785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0629668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0631508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0633379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0635264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0637107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0638969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0640822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0642662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0644551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0646464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0648319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0650165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0652040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0653976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0655815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0657649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0659820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0661699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0663546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0665391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0667215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0669132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0671071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0672898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0674749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0676666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0678597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0680414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0682273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0684111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0685976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0687804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0689658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0691496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0693410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0695284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0697144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0698972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0700872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0702782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0704603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0706461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0708307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0710137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0712012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0713818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0715658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0717536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0719413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0721265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0723109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0724988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0726860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0728715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0730566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0732409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0734240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0736086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0737967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0739785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0741695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0743576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0745433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0747244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0749129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0751015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0752873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0754731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0756579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0758642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0760612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0762440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0764294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0766203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0768132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0769972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0771795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0773707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0775615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0777471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0779309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0781169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0783014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0784856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0786690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0788525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0790402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0792277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0794099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0795948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0797832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0799692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0801531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0803366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0805231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0807036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0808877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0810722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0812585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0814442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0816344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0818193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0820070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0821940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0823846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0825683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0827566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0829400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0831234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0833076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0834920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0836749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0838615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0840526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0842373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0844208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0846082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0847974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0849812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0851647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0853524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0855376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0857224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0859406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0861268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0863200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0865150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0866981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0868839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0870742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0872671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0874492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0876354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0878181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0880034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0881841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0883688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0885524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0887426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0889286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0891136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0892971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0894854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0896739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0898567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0900421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0902267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0904102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0905934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0907792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0909635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0911525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0913407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0915263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0917101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0918984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0920865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0922701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0924532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0926338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0928175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0930006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0931880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0933748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0935645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0937528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0939386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0941195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0943079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0944986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0946845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0948659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0950550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0952385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0954371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0956131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0957969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0960235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0962192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0964045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0965877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0967788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0969687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0971514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0973337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0975201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0977043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0978872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0980704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0982558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0984441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0986329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0988168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0989992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0991893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0993762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0995605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0997439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.0999313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1001138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1002990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1004829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1006696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1008563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1010463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1012290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1014213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1015981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1017872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1019700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1021543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1023381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1025208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1027053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1028896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1030733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1032555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1034455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1036349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1038190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1040017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1041931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1043827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1045665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1047513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1049368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1051216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1053041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1054928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1056751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1059032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1060949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1062789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1064621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1066537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1068417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1070262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1072094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1073960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1075769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1077620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1079452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1081311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1083172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1085049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1086903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1088750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1090704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1092604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1094459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1096312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1098151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1099986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1101828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1103666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1105489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1107354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1109929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1111771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1113611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1115481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1117358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1119216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1121029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1122874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1124699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1126557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1128370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1130200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1132094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1134027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1135837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1137678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1139558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1141462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1143281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1145119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1146935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1148757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1150584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1152402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1154282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1156168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1158043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1161521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1163455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1165411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1167314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1169128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1170982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1172837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1174674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1176502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1178404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1182005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1185632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1189269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1192824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1196474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1200006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1203627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1207178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1210705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1214243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1217756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1221293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1224851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1228390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1231944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1235599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1239154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1242687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1246256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1249840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1253385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1256939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1260930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1264464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1268017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1271550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1275085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1278723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1282342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1285881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1289391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1292972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1296575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1300080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1303562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1307091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1310634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1314147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1317659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1321178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1324765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1328322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1331837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1335350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1338938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1342519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1346030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1349566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1353104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1356689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1360597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1364174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1367721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1371242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1374826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1378433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1381964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1385470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1389040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1392632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1396173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1399686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1403196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1406710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1410249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1413773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1417274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1420848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1424414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1427929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1431424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1434983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1438574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1442102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1445593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1449114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1452651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1456221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1460189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1463742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1467352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1470908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1474395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1477907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1481492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1485062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1488552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1492048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1495584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1499086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1502586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1506101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1509638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1513200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1516740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1520274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1523816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1527401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1530957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1534541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1538095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1541622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1545136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1548652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1552185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1555735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1559728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1563343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1566896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1570421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1574008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1577594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1581128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1584655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1588154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1591687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1595219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1598751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1602261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1605843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1609446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1612960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1616486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1620086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1623695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1627219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1630728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1634230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1637748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1641266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1644761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1648286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1651879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1655502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 18%] 2024-08-08T21:00:08.1659401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1662954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1666560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1670152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1673655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1677172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1680698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1684193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1687690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1691206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1694757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1698269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1701839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1705437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1708947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1712457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1716006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1719564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1723085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1726590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1730057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1733554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1737081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1740585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1744088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1747638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1751266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1754796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1758378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1762299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1765913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1769436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1772958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1776480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1780026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1783557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1787084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1790618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1794243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1797874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1801374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1804899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1808481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1812038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1815546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1819075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1822596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1826109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1829618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1833138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1836727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1840310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1843880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1847391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1850917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1854522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1858072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1862027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1865577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1869116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1872612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1876139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1879677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1883205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1886796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1899801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1903482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1907012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1910630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1914219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1917752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1921267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1924780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1928284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1931794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1935312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1938807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1942363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1945947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1949447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1952947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1956572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1960645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1964174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1967684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1971220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1974768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1978292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1981793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1985313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1988935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1992507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1995998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.1999502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2003078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2006662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2010160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2013665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2017194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2020701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2024189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2027694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2031231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2034729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2038280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2041857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2045394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2048942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2052455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2056075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2059992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2063554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2067066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2070581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2074082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2077574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2081052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2084655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2088258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2091756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2095250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2098811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2102407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2105914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2109417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2112930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2116446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2119925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2123417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2126917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2130499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2134111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2137613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2141128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2144692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2148246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2151730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2155269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2159124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2162636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2166099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2169580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2173093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2176673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2180249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2183751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2187267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2190821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2194375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2197876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2201410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2204938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2208437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2211959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2215481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2219008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2222573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2226146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2229697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2233221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2236754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2240331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2243844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2247343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2250825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2254383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2257917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2261817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2265313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2268921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2272529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2276041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2279531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2283124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2286794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2290287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2293817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2297369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2300941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2304507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2308040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2311604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2315172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2318770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2322367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2325906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2329460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2333011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2336583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2340125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2343689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2347226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2350740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2354328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2357875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2361758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2365383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2369002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2372562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2376103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2379671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2383294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2386869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2390425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2393962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2397515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2401091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2404622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2408150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2411736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2415334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2418849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2422362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2425966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2429553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2433101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2436699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2440243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2443798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2447345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2450880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2454424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2458039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2461989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2465504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2469049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2472709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2476312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2479843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2483381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2486936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2490468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2493997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2497523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2501032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2504614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2508192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2511716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2515263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2518839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2522401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2525912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2529458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2532987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2536491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2540031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2543602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2547114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2550683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2554321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2557897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2561813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2565455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2569082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2572646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2576207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2579749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2583285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2586823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2590346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2593860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2597451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2601097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2604642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2608172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2611762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2615371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2618915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2622428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2625970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2629528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2633064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2636594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2640138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2643727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2647349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2650904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2654501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2658136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2662114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2665670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2669205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2672753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2676274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2679792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2683340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2686886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2690506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2694088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2697641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2701196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2704791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2708383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2711925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2715462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2718992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2722511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2726064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2729623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2733165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2736816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2740399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2743963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2747504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2751080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2754692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2758231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2762096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2765602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2769127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2772683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2776220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2779742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2783425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2786986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2790521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2794032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2797649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2801280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2804818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2808346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2811881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2815445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2819001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2822549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2826157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2829727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2833340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2836872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2840416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2844017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2847594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2851109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2854695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2858270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2862168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2865702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2869251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2872803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2876418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2880025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2883546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2887093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2890725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2894317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2897852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2901417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2904964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2908474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2912020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2915582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2919120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2922697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2926272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2929802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2933301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2936853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2940447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2943993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2947527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2951052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2954597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2958152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2962050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2965576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2969199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2972827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2974650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2976495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2978395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2980343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2982168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2984036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2985879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2987731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2989572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2991410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2993252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2995128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2997004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.2998828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3000678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3002559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3004442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3006262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3008107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3009955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3011785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3013612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3015467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3017309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3019172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3021060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3022904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3024765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3026627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3028527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3030362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3032229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3034104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3035968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3037790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3039641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3041446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3043279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3045155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3047063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3048877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3050703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3052604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3054527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3056361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3058196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3060394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3062245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3064079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3065899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3067740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3069664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3071566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3073384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3075240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3077138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3079029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3080851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3082669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3084512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3086315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3088153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3089977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3091823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3093672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3095562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3097377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3099222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3101071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3102949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3104779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3106645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3108465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3110305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3112142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3113999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3115837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3117708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3119608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3121460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3123301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3125175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3127058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3128885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3130711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3132529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 19%] 2024-08-08T21:00:08.3134367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3136215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3138057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3139873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3141756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3143653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3145468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3147311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3149191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3151089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3152900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3154790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3156635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3158794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3160656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3162515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3164341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3166276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3168157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3170015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3171835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3173743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3175606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3177430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3179279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3181126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3182956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3184782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3186635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3188470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3190340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3192214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3194069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3195913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3197793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3200419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3202269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3204122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3205971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3207805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3209641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3211513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3213334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3215229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3217101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3218953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3220759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3222645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3224524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3226386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3228206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3230056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3231881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3233738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3235556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3237406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3239288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3241173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3243006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3244822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3246711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3248607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3250435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3252276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3254174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3256023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3257861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3260039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3261900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3263809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3265699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3267506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3269349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3271247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3273121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3274952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3276774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3278627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3280431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3282267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3284104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3285964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3287819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3289714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3291553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3293411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3295269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3297167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3298999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3300862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3302693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3304524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3306361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3308196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3310019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3311839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3313737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3315709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3317500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3319320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3321206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3323105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3324933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3326761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3328618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3330458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3332302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3334174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3336017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3337934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3339813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3341669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3343507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3345413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3347276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3349125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3350947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3352796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3354643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3356485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3358377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3360509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3362406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3364306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3366152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3367995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3369881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3371769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3373612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3375458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3377289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3379108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3380950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3382795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3384627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3386498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3388398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3390243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3392082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3393955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3395815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3397660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3399465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3401288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3403109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3404969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3406787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3408624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3410488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3412383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3414195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3416034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3417928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3419834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3421652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3423506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3425345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3427193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3429036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3430879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3432734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3434625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3436520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3438351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3440190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3442063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3443933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3445753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3447606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3449453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3451281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3453103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3454996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3456841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3459086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3461034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3462865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3464719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3466592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3468492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3470324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3472182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3474000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3475839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3477664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3479526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3481338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3483225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3485089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3486970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3488736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3490584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3492467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3494300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3496132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3497953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3499794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3501621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3503447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3505262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3507144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3509043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3510870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3512684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3514588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3516485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3518310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3520140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3521980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3523840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3525655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3527500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3529316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3531203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3533049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3534890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3536704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3538597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3540451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3542279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3544092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3545946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3547757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3549596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3551425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3553257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3555118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3556992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3559287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3561099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3562934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3564852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3566776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3568619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3570458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3572280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3574118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3575952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3577778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3579583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3581490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3583412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3585216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3587057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3588927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3590823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3592636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3594479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3596298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3598151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3599957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3601790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3603612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3605522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3607380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3609214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3611039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3612923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3614797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3616615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3618458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3620285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3622104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3623917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3625759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3627586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3629448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3631304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3633146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3635030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3636913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3638768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3640611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3642446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3644262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3646106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3647939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3649805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3651617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3653506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3655425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3657295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3659444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3661410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3663301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3665140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3666944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3668783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3670598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3672454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3674262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3676085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3677988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3679897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3681726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3683532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3685416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3687323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3689148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3690970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3692824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3694664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3696495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3698323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3700168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3702086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3703970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3705802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3707617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3709515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3711375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3713202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3715030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3716898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3718708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3720555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3722390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3724248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3726049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3727927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3729801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3731657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3733468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3735330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3737224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3739062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3740906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3742739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3744591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3746439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3748269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3750082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3751959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3753838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3755701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3757518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3759764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3761766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3763526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3765353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3767185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3769037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3770849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3772683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3774499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3776410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3778293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3780123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3781953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3783870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3785730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3787573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3789404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3791257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3793067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3794909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3796731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3798552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3800408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3802270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3804109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3805946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3807814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3809667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3811594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3813342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3815166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3816978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3818823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3820665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3822495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3824365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3826269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3828118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3829939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3831826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3833703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3835555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3837382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3839233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3841053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3842891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3844695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3846530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3848394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3850292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3852100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3853938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3855858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3857770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3859927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3861775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3863623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3865451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3867282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3869561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3871879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3873765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3875700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3877538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3879389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3881297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3883207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3885028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3886864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3888698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3890507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3892338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3894171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3896029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3897882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3899766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3901591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3903453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3905307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3907193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3909015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3910860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3912669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3914503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3916330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3918192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3920004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3921826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3923713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3925606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3927431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3929251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3931143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3933007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3934887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3936709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3938546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3940381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3942216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3944032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3945863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3947740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3949617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3951421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3953247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3955189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3957044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3959294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3961172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3963038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3964848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3966696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3968529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3970383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3972304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3974379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3976207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3978055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 20%] 2024-08-08T21:00:08.3979943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3981825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3983650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3985495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3987323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3989143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3990995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3992839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3994664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3996525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.3998415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4000245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4002070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4003926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4005807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4007637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4009465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4011293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4013112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4014964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4016775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4018614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4020478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4022360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4024165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4025996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4027865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4029759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4031555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4033382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4035207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4037053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4038918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4040690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4042506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4044376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4046253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4048071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4049917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4051796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4053737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4055630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4057390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4059641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4061503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4063324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4065159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4066978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4068881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4070763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4072601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4074428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4076232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4078118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4080039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4081883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4083685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4085516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4087337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4089190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4090993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4092816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4094681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4096584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4098384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4100221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4102093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4103978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4105806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4107634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4109466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4111281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4113097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4114910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4116740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4118611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4120491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4122300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4124134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4126003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4127862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4129673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4131510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4133338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4135132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4136954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4138778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4140622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4142462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4144344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4146161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4148009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4149876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4151753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4153551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4155426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4157232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4159432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4161283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4163135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4164932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4166825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4168732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4170565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4172378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4174248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4176161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4177990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4179807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4181628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4183477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4185317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4187151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4188973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4190854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4192732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4194547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4196386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4198239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4200126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4201925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4203790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4205575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4207430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4209237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4211069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4212890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4214729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4216576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4218458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4220357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4222172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4224010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4225885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4227752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4229609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4231451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4233287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4235208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4237072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4238913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4240803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4242712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4244544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4246373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4248236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4250131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4251975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4253810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4255678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4257590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4259763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4261610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4263445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4265363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4267305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4269116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4270965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4272874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4274806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4276623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4278478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4280312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4282176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4284001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4285862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4287679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4289552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4291423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4293242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4295080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4296960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4298834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4300660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4302514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4304346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4306169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4307987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4309833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4311663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4313531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4315492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4317323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4319089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4320947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4322843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4324672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4326538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4328355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4330191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4332011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4333855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4335651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4337535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4339401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4341256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4343064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4344947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4346813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4348662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4350482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4352298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4354142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4356029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4357868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4360085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4362055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4363984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4365823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4367654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4369565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4371475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4373310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4375144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4376997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4378821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4380654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4382474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4384307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4386203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4388069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4389914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4391742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4393643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4395503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4397342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4399173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4401035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4402852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4404694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4406532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4408400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4410261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4412157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4413992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4415836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4417728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4419624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4421461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4423292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4425130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4426955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4428805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4430650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4432484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4434353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4437150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4438995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4440833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4442700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4444596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4446426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4448236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4450172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4451913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4453770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4455640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4457501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4459817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4461788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4463607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4465449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4467364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4469279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4471077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4472903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4474731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4476563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4478384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4480211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4482056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4483888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4485750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4487619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4489463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4491303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4493185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4495062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4496918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4498773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4500615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4502526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4504402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4506158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4507975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4509874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4511744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4513589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4515397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4517287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4519162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4521021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4522842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4524690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4526519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4528375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4530185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4532023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4533889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4535864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4537673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4539491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4541391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4543278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4545117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4546950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4548802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4550639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4552475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4554306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4556179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4558054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4560308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4562143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4563985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4565904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4567804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4569630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4571468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4573322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4575124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4576958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4578783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4580635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4582505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4584414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4586252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4588140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4589978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4591862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4593690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4595551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4597377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4599224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4601044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4602876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4604698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4606556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4608461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4610299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4612123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4613993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4615878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4617707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4619527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4621353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4623192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4625021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4626852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4628670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4630567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4632451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4634273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4636119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4637989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4639890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4641699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4643547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4645374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4647219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4649021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4650856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4652675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4654706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4656473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4658315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4660555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4662500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4664386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4666195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4668031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4669864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4671689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4673500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4675342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4677175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4679053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4680940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4682776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4684604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4686429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4688288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4690160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4691974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4693772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4695596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4697408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4699261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4701056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4702871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4704749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4706639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4708432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4710254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4712131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4714027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4715833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4717680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4719515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4721370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4723182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4725006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4726852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4728729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4730609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4732435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4734270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4736134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4737997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4739807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4741649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4743482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4745296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4747114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4748959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4750791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4752659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4754557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4756387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4758239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4760527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4762444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4764293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4766158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4767977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4769816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4771652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4773520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4775335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4777239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4779129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4780978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4782776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4784653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4786537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4788367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4790197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4792026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4793867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4795703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4797535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4799354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4801234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4803120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4804950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4806768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4808665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4810551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4812380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4814207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4816047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4817903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4819715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 21%] 2024-08-08T21:00:08.4821567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4823389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4825273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4827126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4828972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4830788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4832679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4834586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4836435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4838258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4840119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4841933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4843770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4845587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4847421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4849298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4851196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4853043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4854929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4856810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4859085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4860959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4862793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4864623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4866446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4868280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4870095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4871914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4873718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4875635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4877535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4879332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4881163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4883055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4884965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4886763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4888596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4890424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4892280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4894095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4895943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4897769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4899669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4901539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4903382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4905204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4907106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4908967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4910797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4912642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4914468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4916283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4918094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4919943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4921773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4923635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4925509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4927352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4929187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4931059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4932918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4934755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4936587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4938412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4940229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4942061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4943926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4945739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4947618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4949497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4951355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4953165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4955096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4956956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4959206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4961039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4962868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4964688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4966537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4968346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4970178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4972096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4974019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4975844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4977660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4979561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4981452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4983276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4985092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4986940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4988770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4990593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4992416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4994258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4996135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4998010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.4999837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5001652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5003526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5005369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5007185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5009001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5010859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5012659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5014475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5016301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5018144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5019983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5021865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5023692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5025542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5027392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5029288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5031114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5032959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5034803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5036627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5038475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5040324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5042157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5043977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5045862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5047741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5049554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5051366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5053244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5055172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5057002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5059205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5061092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5062925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5064743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5066579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5068391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5070342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5072223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5074046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5075869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5077800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5079683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5081522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5083347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5085198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5087002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5088841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5090663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5092478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5094337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5096189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5098026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5099860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5101728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5103577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5105408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5107239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5109055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5110860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5112705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5114533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5116347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5118202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5120105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5121939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5123747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5125632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5127499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5129348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5131156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5133001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5134858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5136700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5138504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5140327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5142184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5144084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5145880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5147699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5149558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5151468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5153258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5155101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5156948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5159166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5161028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5162849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5164696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5166620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5168520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5170341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5172180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5174075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5175969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5177792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5179623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5181452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5183246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5185071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5186889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5188738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5190587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5192461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5194282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5196120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5197965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5199850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5201657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5203504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5205306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5207132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5208951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5210807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5212610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5214422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5216306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5218183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5219999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5221813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5223742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5225545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5227367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5229178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5231011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5232832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5234652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5236462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5238293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5240190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5242062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5243871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5245690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5247601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5249451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5251283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5253120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5255030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5256841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5259091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5260961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5262802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5264691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5266599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5268424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5270263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5272119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5273990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5275816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5277650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5279475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5281277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5283117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5284948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5286765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5288612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5290511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5292360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5294196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5296067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5297972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5299818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5301662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5303497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5305326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5307189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5309017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5310868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5312728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5314623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5316433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5318268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5320136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5322036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5323841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5325684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5327516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5329372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5331181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5333028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5334862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5336746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5338646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5340476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5342328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5344215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5346102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5347930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5349779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5351632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5353474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5355347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5357196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5359418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5361348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5363242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5365094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5366932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5368805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5370725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5372555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5374418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5376236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5378085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5379912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5381770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5383590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5385420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5387292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5389199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5391005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5392845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5394722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5396630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5398443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5400274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5402116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5403940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5405761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5407576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5409412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5411292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5413163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5414977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5416816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5418695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5420567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5422386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5424245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5426092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5427927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5429763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5431605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5433470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5435406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5437314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5439146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5441011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5442887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5444786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5446607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5448465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5450273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5452102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5453934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5455838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5457649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5459957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5461900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5463739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5465575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5467456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5469362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5471202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5473042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5474874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5476723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5478569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5480415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5482240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5484125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5486017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5487863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5489695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5491581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5493458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5495260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5497104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5498933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5500787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5502601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5504452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5506281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5508182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5510043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5511886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5513710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5515605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5517470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5519303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5521134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5522993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5524806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5526631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5539051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5541031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5542950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5544841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5546665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5548495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5550414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5552215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5554053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5555957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5557803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5560062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5561939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5563800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5565604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5567520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5569437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5571310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5573130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5575051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5576963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5578836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5580654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5582516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5584347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5586210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5588042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5589898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5591767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5593651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5595480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5597296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5599197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5601095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5602925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5604759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5606615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5608458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5610289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5612112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5613965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5615840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5617718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5619551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5621402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5623305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5625177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5627021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5628849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5630716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5632536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5634384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5636208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5638058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5639902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5641784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5643606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5645460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5647270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5649158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5651033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5652902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5654764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5656593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5658811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5660718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5662593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5664372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5666324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5668246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5670077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5671903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5673810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 22%] 2024-08-08T21:00:08.5676307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5678145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5679982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5681824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5683652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5685484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5687304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5689124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5691028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5692903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5694738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5696562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5698458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5700311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5702137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5703966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5705822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5707630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5709471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5711413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5713177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5715042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5716952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5718788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5720632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5722509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5724387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5726221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5728049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5729871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5731682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5733521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5735420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5737251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5739119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5741016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5742850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5744677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5746536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5748408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5750255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5752066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5753901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5755763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5757623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5759873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5761750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5763667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5765599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5767412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5769248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5771126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5773031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5774825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5776655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5778482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5780308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5782128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5783949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5785786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5787661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5789534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5791351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5793195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5795078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5796960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5798783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5800637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5802487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5804319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5806145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5808007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5809857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5811727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5813623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5815451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5817285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5819135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5821016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5822839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5824776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5826511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5828354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5830181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5832044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5833857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5835694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5837570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5839480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5841293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5843119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5845017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5846918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5848755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5850585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5852439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5854283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5856166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5858004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5860264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5862214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5864119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5865931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5867777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5869674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5871548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5873390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5875224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5877084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5878896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5880730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5882561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5884411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5886303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5888195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5890032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5891892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5893757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5895663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5897493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5899366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5901182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5903008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5904852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5906688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5908511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5910367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5912260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5914103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5915931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5917796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5919688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5921525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5923355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5925177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5927007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5928844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5930672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5932481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5934350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5936258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5938078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5939915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5941783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5943680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5945480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5947326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5949146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5950986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5952781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5954612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5956476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5958735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5960713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5962556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5964387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5966286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5968193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5970006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5971850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5973694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5975527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5977363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5979228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5981080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5982959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5984839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5986701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5988540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5990434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5992318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5994153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5995981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5997789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.5999632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6001460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6003324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6005143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6007028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6008905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6010766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6012580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6014463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6016347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6018194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6020003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6021853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6023695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6025547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6027392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6029228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6031072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6032956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6034893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6036726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6038564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6040443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6042313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6044126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6045977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6047819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6049646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6051470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6053323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6055195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6057065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6059440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6061299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6063147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6065060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6066963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6068789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6070655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6072467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6074300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6076123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6077989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6079799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6081698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6083601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6085443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6087249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6089109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6090994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6092830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6094654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6096466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6098297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6100130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6101956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6103760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6105641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6107529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6109365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6111193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6113093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6114982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6116816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6118653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6120496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6122379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6124195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6126043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6127868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6129754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6131608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6133447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6135268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6137166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6139020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6140860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6142689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6144554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6146360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6148194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6150023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6151853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6153722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6155627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6157483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6159752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6161701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6163605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6165441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6167279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6169109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6170941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6172774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6174597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6176418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6178303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6180188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6182038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6183842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6185720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6187585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6189424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6191248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6194801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6198323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6201858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6205373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6208876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6212418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6216014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6219597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6223112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6226646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6230235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6233808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6237374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6240828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6244352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6247849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6251361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6254915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6258822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6262497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6266086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6269610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6273140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6276804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6280290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6283792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6287338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6290871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6294367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6297897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6301455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6304982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6308524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6312105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6315643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6319179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6322746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6326381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6329888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6333400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6336967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6340474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6344004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6347527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6351036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6354585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6358209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6362203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6365707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6369312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6372917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6376413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6379894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6383417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6386952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6390464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6393974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6397498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6401087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6404680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6408191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6411692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6415257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6418808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6422281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6425770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6429289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6432789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6436270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6439774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6443293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6446769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6450314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6453884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6457466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6461475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6465076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6468672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6472205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6475752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6479280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6482800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6486365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6489894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6493411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6497002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6500602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6504113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6507609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6511186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6514788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6518289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6521822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6525338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6528869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6532380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6535895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6539409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6542974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6546534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6550042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6553555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6557189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6561234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6564735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6568254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6571790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6575307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6578812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6582324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6585829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6589393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6592946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6596451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6599970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6603547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6607123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6610603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6614122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6617639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6621122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6624633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6628161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6631672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6635259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6638832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6642379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6645895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6649458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6653025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6656592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6660546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6664066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6667570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6671070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6674573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6678070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6681614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6685270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6688740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6692219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6695721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6699319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6702896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6706372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6709868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6713388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6716900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6720391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6723916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6727418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6730971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6734517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6738022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6741546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6745110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6748666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6752150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6755702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6759808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6763334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6766826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6770358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6773855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6777414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6780991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6784510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6788003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6791553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6795131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6798643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6802160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6805662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6809176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6812712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 23%] 2024-08-08T21:00:08.6816230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6819738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6823293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6826877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6830404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6833910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6837470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6841038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6844502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6847980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6851474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6855036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6858954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6862505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6866013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6869612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6873210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6876711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6880202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6883795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6887383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6890855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6894370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6897916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6901438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6904947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6908469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6912020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6915510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6919062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6922639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6926141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6929639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6933160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6936783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6940290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6943804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6947302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6950802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6954319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6957876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6961763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6965360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6968949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6972454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6975976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6979508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6983163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6986580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6990079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6993589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.6997108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7000614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7004117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7007591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7011133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7014668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7018146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7021639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7025193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7028749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7032195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7035688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7039204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7042691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7046170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7049681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7053190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7056787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7060764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7064299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7067827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7071487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7075090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7078587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7082122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7085652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7089151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7092654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7096179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7099672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7103199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7106776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7110299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7113801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7117306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7120872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7124414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7127924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7131429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7134936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7138509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7141970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7145462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7148954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7152538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7156162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7160049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7163591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7167205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7170785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7174277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7177772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7181270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7184748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7188224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7191727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7195224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7198783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7202342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7205840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7209341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7212900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7216401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7219898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7223409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7226906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7230383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7233887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7237478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7240975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7244535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7248100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7251617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7255163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7259152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7262761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7266243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7269731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7273208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7276703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7280218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7283717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7287172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7290728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7294312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7297823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7301298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7304855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7308427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7311902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7315387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7318905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7322426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7325957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7329448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7332942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7336458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7340028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7343582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7347072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7350575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7354098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7357655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7361545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7365071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7368579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7372049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7375546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7379037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7382519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7386082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7389669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7393183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7396662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7400204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7403752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7407261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7410759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7414250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7417740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7421255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7424741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7428227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7431763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7435307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7438786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7442242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7445805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7449318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7452790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7456318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7460182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7463719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7467214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7470674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7474180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7477692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7481273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7484839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7488354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7491887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7495457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7499025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7502538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7506070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7509583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7513082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7516591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7520078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7523542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7527072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7530620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7534126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7537695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7541210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7544762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7548280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7551776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7555300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7559215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7562808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7566339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7569858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7573502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7577822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7581366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7584895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7588490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7592120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7595655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7599183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7602728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7606260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7609773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7613270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7616792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7620390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7623962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7627482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7631004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7634587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7638135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7641643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7645168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7648723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7652243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7655818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7659773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7663376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7667006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7670614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7674169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7677726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7681326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7684904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7688430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7691969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7695480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7698979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7702500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7706052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7709564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7713128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7716705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7720254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7723768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7727328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7730911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7734449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7737974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7741496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7745027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7748579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7752118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7755658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7759665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7763318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7766865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7770380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7773967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7777551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7781063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7784566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7788083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7791629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7795150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7798661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7802166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7805693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7809261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7812854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7816376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7819919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7823496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7827043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7830582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7834140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7837738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7841267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7844814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7848346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7851867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7855487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7859456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7863027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7866554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7870149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7873719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7877262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7880788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7884307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7887826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7891371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7894938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7898510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7902480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7906084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7909596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7913094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7916690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7920277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7923780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7927290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7930804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7934339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7937854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7941369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7944877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7948417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7951958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7955484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7959369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7963015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7966601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7970074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7972010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7973742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7975571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7977354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7979177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7981001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7982879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7984755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7986593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7988415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7990282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7992135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7993962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7995784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7997603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.7999412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8001215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8003042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8004831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8006683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8008538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8010372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8012156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8014007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8015863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8017695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8019482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8021302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8023118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8024958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8026761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8028594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8030458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8032355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8034171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8035972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8037842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8039715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8041522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8043337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8045169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8046986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8048795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8050594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8052423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8054236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8056140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8058000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8060221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8062335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8064204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8066108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8067924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8069820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8071586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8073436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8075235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8077112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8078891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8080805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8082685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8084523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8086378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8088233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8090062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8091872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8093732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8095479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8097298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8099124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8100960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8102736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8104602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8106550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8108287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8110089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8111953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8113817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8115633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8117437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8119268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8121096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8122900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8124732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8126538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8128435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8130281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8132146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8133924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8135859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8137692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8139500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8141308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8143154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8144946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8146768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8148582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8150422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8152251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8154095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8155960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8157792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8160093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8162002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8163830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8165657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8167474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8169277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8171106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8172933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8174751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8176696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8178511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8180324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8182120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8183958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8185821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8187650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8189441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8191255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8193058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8194890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8196683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8198504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8200303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8202181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8204018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 24%] 2024-08-08T21:00:08.8205828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8207633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8209512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8211364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8213169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8214995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8216812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8218624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8220421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8222237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8224038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8225881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8227749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8229517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8231328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8233171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8235005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8236828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8238639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8240442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8242231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8244036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8245877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8247672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8249535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8251396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8253239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8255036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8256954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8259198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8261073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8262875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8264703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8266495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8268316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8270106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8271896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8273832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8275737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8277553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8279351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8281238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8283122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8284930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8286738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8288576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8290405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8292216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8294018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8295852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8297723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8299589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8301405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8303215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8305109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8306957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8308773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8310581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8312413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8314202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8316018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8317831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8319677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8321511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8323372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8325187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8327020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8328861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8330727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8332529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8334353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8336171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8337976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8339809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8341640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8343451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8345258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8347128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8349001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8350809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8352621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8354487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8356384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8358188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8360385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8362224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8364045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8365848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8367666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8369465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8371381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8373245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8375057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8376862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8378757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8380617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8382420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8384236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8386077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8387863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8389686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8391501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8393314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8395167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8397030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8398852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8400657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8402505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8404338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8406151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8408066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8409773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8411557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8413373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8415185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8416993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8418818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8420690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8422510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8424305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8426166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8428027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8429864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8431663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8433495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8435352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8437200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8439001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8440825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8442673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8444543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0004s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8446331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8448137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8449982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8451859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8453664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8455505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8457334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8459552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8461392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8463196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8465088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8466937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8468833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8470640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8472462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8474282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8476159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8478030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8479859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8481697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8483493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8485322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8487129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8488956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8490731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8492591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8494459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8496289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8498077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8499941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8501792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8503629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8505425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8507254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8509047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8510880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8512679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8514510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8516336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8518207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8520022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8521825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8523701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8525564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8527380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8529204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8531003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8532812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8534614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8536405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8538222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8540077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8541916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8543733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8545542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8547414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8549247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8551055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8552874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8554720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8556564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8558772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8560598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8562445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8564324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8566227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8568036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8569877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8571738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8573624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8575438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8577244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8579048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8580847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8582671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8584496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8586300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8588141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8590012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8591827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8593625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8595461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8597337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8599152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8600956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8602754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8604564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8606402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8608196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8610075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8611815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8613689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8615569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8617461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8619566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8621375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8623187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8624972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8626768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8628586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8630354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8632157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8633945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8635753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8637561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8639407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8641191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8642990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8644822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8646688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8648501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8650310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8652108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8653890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8655747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8657563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8659726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8661632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8663509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8665305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8667090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8668939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8670797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8672611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8674466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8676197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8678043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8679793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8681567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8683366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8685201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8687073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8688853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8690654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8692492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8694365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8696137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8697944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8699743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8701552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8703348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8705330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8707135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8709010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8710858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8712649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8714470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8716367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8718213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8719913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8721716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8723584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8725357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8727090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8728880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8730782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8732483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8734389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8736213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8738102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8739798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8741677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8743505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8745382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8747079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8748880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8750728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8752519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8754237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8756064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8757893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8760137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8761948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8763721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8765606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8767465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8769250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8771042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8772837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8774633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8776463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8778219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8780037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8781907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8783802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8785570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8787360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8789227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8791051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8792852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8794639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8796457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8798226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8800024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8801819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8803639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8805492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8807311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8809100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8810910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8812723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8814576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8816362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8818204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8819962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8821746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8823600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8825401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8827174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8829004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8830858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8832663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8834460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8836243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8838087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8839927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8841710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8843482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8845289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8847090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8848869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8850663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8852448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8854316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8856179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8857977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8860175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8862118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8863973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8865785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8867592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8869422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8871198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8873006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8874811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8876611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8878472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8880324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8882135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8883929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8885762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8887587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8889395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8891198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8892994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8894773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8896577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8898379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8900167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8901991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8903842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8905645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8907416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8909259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8911096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8912908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8914682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8916499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8918289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8920109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8921888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8923689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8925505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8927363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8929133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8930911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8932732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8934598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8936375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8938157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8951318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8953414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8955103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8956994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8959252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8961092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8963021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8964897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8966714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8968521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8970390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8972255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8974061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8975871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8977665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8979447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8981250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8983042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8984815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8986627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8988487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8990289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8992065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8993905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8996464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.8998280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9000053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9001849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9003628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9005445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9007221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9009014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9010848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9012719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9014494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9016302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9018144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9019994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9021783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9023563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9025368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9027154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9028933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9030716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9032526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9034434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9036290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9038071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9039869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9041727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9043571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9045344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9047145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 25%] 2024-08-08T21:00:08.9048942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9050710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9052499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9054287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9056148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9057969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9060197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9062015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9063829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9065726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9067623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9069392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9071196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9072968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9074761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9076545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9078342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9080123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9081896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9083771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9085625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9087406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9089180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9091036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9092889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9094692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9096477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9098298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9100098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9101892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9103684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9105497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9107366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9109206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9111018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9112802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9114665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9116486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9118273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9120068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9121889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9123654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9125448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9127237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9129051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9130860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9132731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9134519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9136310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9138135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9139960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9141765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9143568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9145374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9147156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9148958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9150857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9152566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9154393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9156284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9158086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9160539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9162551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9164453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9166245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9168013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9169810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9171587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9173399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9175178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9176976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9178745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9180620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9182464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9184260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9186049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9187903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9189723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9191527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9193317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9195117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9196900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9198675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9200474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9202263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9204085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9205898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9207699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9209495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9211326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9213150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9214951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9216746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9218531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9220297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9222104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9223902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9225668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9227497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9229339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9231153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9232920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9234771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9236610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9238422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9240190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9241990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9243761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9245563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9247325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9249106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9250942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9252832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9254627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9256390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9258227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9260668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9262348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9264121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9265923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9267716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9269498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9271273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9273078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9274859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9276744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9278611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9280389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9282203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9284036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9286210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9287983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9289781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9291537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9293313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9295090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9296904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9298669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9300500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9302328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9304139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9305903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9307735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9309567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9311366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9313172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9314940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9316745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9318541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9320338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9322120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9323967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9325817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9327609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9329390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9331223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9333121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9334898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9336671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9338460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9340273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9342041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9343830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9345616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9347469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9349287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9351084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9352946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9354853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9356711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9359188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9361121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9363052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9365005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9366739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9368562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9370396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9372321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9374213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9376047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9377861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9379726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9381599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9383428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9385254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9387082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9388902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9390752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9392565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9394380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9396227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9398113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9399943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9401750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9403621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9405502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9407349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9409159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9410997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9412820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9414683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9416482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9418316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9420162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9422048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9423843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9425664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9427533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9429432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9431228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9433035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9434870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9436697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9438517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9440330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9442157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9444020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9445885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9447696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9449524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9451394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9453315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9455137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9456962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9459221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9461083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9462901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9464734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9466546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9468417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9470310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9472135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9473973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9475834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9477733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9479546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9481370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9483172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9485001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9486817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9488721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9490474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9492303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9494167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9496074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9497884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9499750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9501635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9503471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9505294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9507113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9508943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9510767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9512584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9514389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9516223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9518088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9519968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9521775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9523650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9525529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9527351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9529160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9530983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9532832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9534636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9536469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9538297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9540151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9542008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9543893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9545709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9547606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9549467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9551295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9553146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9555001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9556800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9559003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9560858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9562687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9564508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9566418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9568331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9570151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9572028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9573903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9575729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9577548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9579369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9581174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9582984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9584811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9586622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9588425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9590299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9592178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9593978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9595808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9597654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9599528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9601319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9603134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9604947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9606786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9608582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9610404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9612203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9614095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9615938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9617761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9619573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9621457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9623303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9625136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9626963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9628796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9630628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9632447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9634284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9636110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9637971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9639835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9641659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9643474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9645323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9647171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9649002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9650829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9652691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9654508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9656349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9658173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9660359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9662286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9664176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9666011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9667812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9669701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9671581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9673427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9675314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9677065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9678884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9680736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9682536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9684477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9686308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9688083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9689862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9691659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9693533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9695409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9697222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9699039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9700871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9702694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9704513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9706321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9708142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9710003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9711873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9713670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9715501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9717406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9719277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9721100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9722912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9724758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9726561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9728391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9730190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9732014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9733845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9735709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9737519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9739348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9741184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9743050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9744935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9746725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9748500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9750321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9752199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9754011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9755816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9757674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9759969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9761826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9763644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9765547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9767458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9769282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9771101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9772908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9774721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9776534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9778334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9780118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9782003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9783891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9785680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9787503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9789368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9791250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9793043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9794874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9796675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9798500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9800290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9802110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9803918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9805785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9807627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9809456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9811262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9813139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9814983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9816793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9818607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9820417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9822209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9824000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9825823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9827627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9829472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9831323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9833146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9834958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9836808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9838649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9840480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9842298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9844111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9845916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9847730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9849582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9851376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9853294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9855159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9857008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9859192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9861167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9863016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9864848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9866646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9868467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9870271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9872110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9873909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9875726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9877590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9879498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 26%] 2024-08-08T21:00:08.9881311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9883116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9884995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9886868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9888677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9890492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9892334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9894160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9895979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9897804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9899630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9901494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9903361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9905176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9906994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9908851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9910686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9912495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9914307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9916152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9917947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9919772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9921586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9923422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9925252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9927115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9928930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9930763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9932611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9934484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9936294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9938136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9939949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9941771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9943582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9945412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9947239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9949092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9950956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9952811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9954627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9956469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9958717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9960548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9962368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9964179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9966002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9967816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9969630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9971437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9973357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9975245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9977039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9978854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9980728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9982630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9984418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9986243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9988056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9989882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9991674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9993500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9995297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9997162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:08.9998995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0000810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0002613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0004488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0006317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0008115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0009941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0011764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0013566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0015359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0017197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0019027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0020882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0022738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0024581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0026400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0028258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0030119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0031949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0033844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0035596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0037418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0039217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0041049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0042836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0044689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0046550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0048396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0050189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0052101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0053979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0055809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0057596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0059890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0061716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0063556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0065358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0067183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0069087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0070994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0072816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0074629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0076532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0078429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0080247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0082063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0083895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0085707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0087515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0089314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0091146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0093009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0094910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0096727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0098549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0100411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0102252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0104069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0105875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0107713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0109508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0111334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0113152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0114996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0116834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0118703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0120515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0122347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0124183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0126067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0127854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0129674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0131462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0133257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0135112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0136904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0138718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0140513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0142379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0144244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0146045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0147842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0149733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0151601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0153460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0155277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0157118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0159324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0161183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0163004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0164820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0166752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0168631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0170458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0172257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0174147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0175999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0177818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0179632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0181469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0183290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0185096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0186902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0188731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0190569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0192515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0194236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0196049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0197895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0199745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0201571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0203399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0205214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0207017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0208846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0210670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0212484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0214333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0216193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0217990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0219787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0221615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0223482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0225296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0227088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0228902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0230786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0232536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0234327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0236136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0237974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0239879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0241662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0243483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0245335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0247212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0249003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0250831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0252686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0254524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0256321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0258135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0260325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0262231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0264106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0265906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0267730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0269606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0271479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0273278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0275113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0276927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0278731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0280527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0282354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0284155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0286008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0287861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0289683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0291524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0293361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0295293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0297037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0298879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0300679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0302504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0304307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0306139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0307915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0309765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0311617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0313453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0315244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0317112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0318964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0320777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0322593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0324403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0326216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0328022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0329839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0331636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0333520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0335415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0337207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0339011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0340882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0342746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0344556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0346350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0348170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0349989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0351738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0353592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0355401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0357278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0359514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0361368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0363167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0365083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0366942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0368743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0370543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0372378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0374171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0375991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0377810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0379651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0381507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0383378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0385211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0387023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0388884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0390742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0392549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0394354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0396162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0397951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0399773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0401584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0403386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0405224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0407096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0408914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0410696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0412547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0414432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0416273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0418076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0419913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0421734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0423583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0425396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0427227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0429083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0430984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0432796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0434630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0436481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0438368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0440154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0441975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0443799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0445632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0447449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0449263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0451093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0453002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0454876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0456688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0458875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0460832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0462738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0464569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0466395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0468237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0470079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0471896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0473741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0475578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0477479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0479398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0481217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0483058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0484897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0486773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0488586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0490433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0492245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0494086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0495904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0497755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0499556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0501418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0503626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0505469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0507262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0509133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0511022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0512855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0514674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0516492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0518328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0520159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0521989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0523800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0525682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0527527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0529339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0531131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0532999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0534873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0536683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0538486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0540302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0542143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0543939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0545770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0547592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0549475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0551324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0553212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0555035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0556932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0559164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0561033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0562852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0564710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0566520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0568353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0570194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0572010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0573888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0575772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0577607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0579431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0581308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0583186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0585019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0586846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0588663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0590471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0592306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0594135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0595956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0597816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0599719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0601557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0603383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0605252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0607134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0608981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0610793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0612630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0614440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0616276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0618070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0619892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0621751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0623672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0625443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0627263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0629115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0631008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0632802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0634634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0636447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0638265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0640080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0641892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0643726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0645600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0647481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0649322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0651127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0653042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0654917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0656721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0658912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0660803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0662612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0664413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0666247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0668071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0669857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0671772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0673665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0675504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0677296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0679187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0681068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0682915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0684725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0686563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0688387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0690248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0692060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0693886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0695759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0697674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0699482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0701300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0703175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0705045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0706853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0708661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0710498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0712333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0714159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0715974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 27%] 2024-08-08T21:00:09.0717799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0719665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0721534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0723343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0725175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0727040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0728902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0730721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0732548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0734382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0736187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0738018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0739835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0741673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0743518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0745398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0747199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0749093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0750861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0752762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0754582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0756431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0758228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0760442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0762263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0764103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0765887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0767777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0769678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0771491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0773301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0775167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0777081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0778907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0780727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0782536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0784370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0786196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0788015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0789822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0791680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0793541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0795368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0797136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0798992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0800878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0802669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0804493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0806318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0808134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0809929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0811751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0813556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0815443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0817291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0819116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0820932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0822839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0824658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0826491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0828317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0830143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0831962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0833773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0835591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0837397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0839246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0841091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0842913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0844728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0846577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0848462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0850288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0852154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0853973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0855773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0857598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0859803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0861620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0863527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0865418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0867260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0869077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0870953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0872825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0874668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0876475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0878305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0880095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0881913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0883701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0885507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0887348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0889227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0891024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0892854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0894709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0896572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0898355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0900161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0902002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0903822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0905632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0907450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0909277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0911142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0913008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0914832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0916668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0918539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0920409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0922242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0924034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0925864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0927657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0929477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0931298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0933132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0934975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0936851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0938703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0940509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0942347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0944234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0946030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0947877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0949676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0951493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0953356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0955195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0957008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0959316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0961270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0963101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0964923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0966806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0968689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0970500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0972306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0974099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0975920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0977736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0979559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0981360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0983227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0985095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0986892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0988712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0990562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0992454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0994244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0996074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0997895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.0999733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1001533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1003367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1005181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1007067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1008913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1010746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1012543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1014415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1016245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1018038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1019868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1021694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1023502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1025311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1027137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1028960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1030809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1032659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1034485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1036296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1038262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1039987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1041816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1043636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1045452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1047258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1049065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1050902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1052740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1054623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1056465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1058293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1060478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1062381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1064251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1066081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1067876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1069703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1071507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1073346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1075140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1076957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1078835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1080735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1082554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1084358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1086235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1088120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1089935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1091803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1093593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1095417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1097238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1099054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1100879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1102753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1104587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1106394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1108218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1110079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1111926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1113745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1115554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1117393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1119193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1121012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1122830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1124677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1126517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1128394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1130214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1132064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1133905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1135797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1137611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1139456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1141261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1143093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1144900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1146717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1148522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1150358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1152275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1154108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1155915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1157764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1160010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1161850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1163665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1165470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1167293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1169099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1170907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1172707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1174620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1176511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1178310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1180123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1181995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1183899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1185696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1187520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1189322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1191146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1192928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1194741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1196543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1198411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1200247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1202072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1203868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1205720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1207577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1209371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1211203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1213024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1214836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1216646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1218483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1220312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1222126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1223980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1225867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1227687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1229505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1231404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1233279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1235092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1236904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1238701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1240510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1242348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1244141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1245962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1247813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1249696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1251482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1253350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1255198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1257079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1259272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1262839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1266363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1269892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1273399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1276916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1280368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1283997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1287600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1291069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1294561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1298125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1301643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1305102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1308607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1312126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1315612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1319071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1322560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1326042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1329580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1333109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1336590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1340106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1343644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1347158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1350658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1354231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1357837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1361596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1365216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1368641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1372118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1375701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1379280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1382778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1386254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1389769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1393314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1396805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1400295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1403773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1407255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1410764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1414258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1417708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1421245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1424790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1428278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1431753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1435327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1438876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1442361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1445856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1449365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1452919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1456431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1460304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1463813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1467377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1470935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1474413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1477900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1481478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1485037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1488478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1491970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1495489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1498964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1502440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1505936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1509438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1512943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1516460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1519952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1523461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1526997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1530526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1534004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1537490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1540978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1544446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1547920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1551407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1554919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1560488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1564137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1567644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1571123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1574663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1578212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1581705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1585158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1588620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1592092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1595591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1599081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1602563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1606105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1609657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1613154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1616650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1620195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1623751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1627239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1630704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1634194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1637685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1641164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1644626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1648118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1651670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1655280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1659126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1662654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1666351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1669839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1673303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1676789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1680262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1683734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1687204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1690690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1694201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1697753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1701299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1704789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1708284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1711818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1715366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1718840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1722377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1725774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1729223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1732687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1736183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1739660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1743113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1746636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1750167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1753707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1757159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1761139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1764724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1768202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1771662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1775155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1778686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1782158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1785642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1789122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1792700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1796252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1799721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1803207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1806739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1810250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1813709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1817190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 28%] 2024-08-08T21:00:09.1820679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1824153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1827637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1831120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1834609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1838132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1841623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1845103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1848597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1852139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1855704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1859578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1863128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1866595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1870056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1873552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1877054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1880531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1884083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1887635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1891094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1894556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1898083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1901613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1905106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1908566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1912008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1915468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1918950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1922409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1925864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1929380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1932965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1936452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1939893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1943434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1946985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1950469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1953985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1957458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1961353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1964842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1968324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1971806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1975370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1978923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1982368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1985842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1989409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1992960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1996427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.1999996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2003410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2006875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2010343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2013832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2017332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2020872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2024381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2027852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2031354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2034911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2038415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2041901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2045411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2048897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2052350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2055878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2059767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2063269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2066730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2070290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2073875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2077330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2080792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2084338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2087908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2091390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2094858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2098305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2101796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2105274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2108735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2112210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2115763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2119301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2122743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2126223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2129775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2133791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2137286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2140754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2144236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2147671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2151126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2154718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2158101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2162128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2165675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2169130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2172614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2176145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2179660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2183128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2186627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2190107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2193551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2197050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2200567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2204042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2207585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2211097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2214589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2218075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2221599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2225124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2228594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2232060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2235532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2238984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2242485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2245968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2249429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2253019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2256573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2260515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2263979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2267561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2271118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2274571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2278028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2281524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2284981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2288461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2291934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2295407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2298958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2302515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2305989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2309435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2312956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2316454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2319892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2323352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2326838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2330301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2333732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2337192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2340679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2344173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2347670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2351126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2354618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2358134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2362032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2365498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2368995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2372479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2375944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2379397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2382903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2386389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2389853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2393412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2406533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2410108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2413497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2417105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2420678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2424158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2427619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2431085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2434569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2438039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2441484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2444978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2448493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2452019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2455543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2459507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2463164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2466708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2470189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2473675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2477167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2480646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2484119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2487611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2491063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2494594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2498110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2501571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2505058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2508571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2512073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2515538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2519034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2522493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2525937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2529409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2532891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2536338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2539842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2543356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2546847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2550307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2553861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2557391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2561373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2564776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2568262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2571685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2575151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2578602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2582022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2585593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2589156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2592625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2596068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2599589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2603133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2606570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2610010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2613486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2616971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2620455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2623907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2627376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2630930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2634457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2637931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2641408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2644960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2648489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2651929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2655469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2659318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2662817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2666257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2669727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2673207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2676654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2680206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2683753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2687236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2690707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2694224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2697786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2701279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2704806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2708261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2711771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2715317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2718828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2722306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2725873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2729448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2732945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2736439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2739989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2743555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2747019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2750488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2754048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2757563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2761523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2765023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2768495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2772109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2775692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2779184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2782679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2786265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2789822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2793296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2796809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2800340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2803850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2807345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2810878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2814400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2817950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2821504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2825006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2828501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2832046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2835571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2839082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2842584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2846090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2849595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2853125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2856646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2860495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2864067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2867637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2871148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2874635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2878210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2881798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2885313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2888810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2892300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2895787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2899306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2902800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2906276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2909808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2913336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2916820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2920299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2923827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2927387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2930869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2934330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2937827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2941316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2944801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2948283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2951787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2955404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2959293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2962820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2966329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2969964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2973576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2977082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2980605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2984130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2987639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2991169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2994646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.2998179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3001711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3005240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3008749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3012266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3015815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3019347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3022842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3026362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3029871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3033335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3035168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3036998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3038813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3040670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3042562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3044397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3046221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3048084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3049969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3051796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3053643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3058151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3062830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3064660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3066472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3068318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3070155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3071993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3073788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3075610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3077528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3079443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3081242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3083084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3084876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3086788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3088670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3090474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3092309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3094126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3095935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3097748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3099571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3101452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3103323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3105104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3106915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3108717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3110597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3112429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3114255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3116176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3117868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3119686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3121489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3123314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3125141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3127011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3128828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3130661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3132453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3134315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3136180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3138026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3139842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3141676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3143492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3145345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3147147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3149001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3150863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3152679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3154532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3156335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3158229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3160524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3162340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3164150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3165979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3167795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3169664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3171413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3173322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3175207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3177031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3178819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3180626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3182536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3184404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3186231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3188048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3189882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3191682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3193506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3195316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3197152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3198974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3200782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3202586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3204418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3206266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3208123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3209927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3211747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3213547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3215341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3217258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3218993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3220840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3222686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3224515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3226342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3228153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3230013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3231887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3233711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3235551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3237354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3239147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3240954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3242744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 29%] 2024-08-08T21:00:09.3244550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3246394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3248278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3250157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3251883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3253775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3255656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3257443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3259632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3261464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3263300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3265095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3266909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3268725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3270654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3272524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3274334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3276158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3278045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3279927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3281733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3283543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3285344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3287138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3288935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3290757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3292573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3294425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3296262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3298085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3299898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3301749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3303605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3305434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3307245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3309031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3310837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3312635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3314470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3316258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3318118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3319973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3321821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3323592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3325549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3327324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3329144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3330923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3332733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3334518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3336329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3338130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3339922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3341784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3343643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3345440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3347230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3349097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3350958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3352763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3354608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3356444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3358265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3360500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3362338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3364175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3366065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3367934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3369763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3371556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3373456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3375328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3377115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3378926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3380781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3382557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3384372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3386176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3388006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3389829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3391704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3393524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3395367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3397166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3399004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3400881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3402705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3404536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3406339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3408167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3409992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3411804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3413653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3415520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3417326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3419125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3420926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3422792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3424665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3426479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3428296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3430079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3431934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3433702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3435518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3437366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3439247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3441043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3442849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3444657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3446551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3448396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3450224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3452038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3453923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3455722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3457556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3459737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3461663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3463543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3465341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3467165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3469000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3470865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3472732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3474552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3476367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3478172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3479961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3481786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3483595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3485443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3487280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3489083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3490947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3492698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3494560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3496416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3498251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3500061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3501858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3503643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3505457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3507233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3509030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3510866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3512743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3514525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3516335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3518181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3520042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3521845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3523647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3525498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3527284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3529088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3530881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3532705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3534568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3536426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3538240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3540063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3541929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3543779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3545581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3547403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3549203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3550977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3552787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3554635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3556469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3558299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3560568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3562376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3564199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3566069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3567954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3569761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3571594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3573390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3575193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3577002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3578844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3580643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3582513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3584397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3586229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3588041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3589890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3591749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3593556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3595349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3597139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3598974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3600785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3602589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3604388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3606260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3608111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3609919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3611721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3613564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3615435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3617220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3619024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3620828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3622671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3624454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3626277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3628080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3629950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3631788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3633602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3635390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3637244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3639064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3640861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3642651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3644462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3646254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3648043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3649857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3651664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3653552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3655400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3657222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3659465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3661310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3663196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3665095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3666915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3668734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3670544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3672370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3674196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3675992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3677871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3679733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3681551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3683328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3685135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3686983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3688856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3690641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3692455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3694260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3696090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3697880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3699698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3701557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3703430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3705217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3707017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3708836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3710723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3712578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3714384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3716211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3718023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3719829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3721623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3723429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3725271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3727114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3728894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3730714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3732524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3734374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3736209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3738006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3739832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3741616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3743417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3745218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3747046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3748818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3750655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3752507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3754371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3756164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3758025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3760237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3762086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3763887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3765701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3767491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3769288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3771084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3772868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3774764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3776651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3778454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3780248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3782115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3784652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3786454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3788250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3790073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3791872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3793677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3795499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3797282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3799158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3801011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3802835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3804635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3806511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3808347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3810159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3812003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3813767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3815550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3817349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3819152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3821002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3822800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3824659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3826453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3828259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3830092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3831924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3833732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3835533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3837326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3839111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3840949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3842737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3844540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3846363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3848229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3850041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3851842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3853700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3855543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3857404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3859548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3861384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3863178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3864990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3866771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3868579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3870454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3872345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3874121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3875915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3877714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3879606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3881450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3883258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3885071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3886906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3888691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3890500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3892321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3894180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3896026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3897822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3899632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3901436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3903290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3905129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3906936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3908836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3910523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3912312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3914141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3915919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3917738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3919590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3921390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3923217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3924999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3926843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3928690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3930520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3932310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3934123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3935925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3937757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3939540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3941356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3943183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3945045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3946819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3948602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3950458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3952313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3954162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3955964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3957780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3960004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3961835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3963631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3965449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3967346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3969245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3971040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3972867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3974903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3976783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3978582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3980392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3982219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3984027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3985816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3987608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3989424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3991239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3993094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3994889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3996727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.3998526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4000379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4002169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4003990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4005768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4007577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4009373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4011177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4012965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4014915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4016647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4018459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4020253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4022116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4023974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4025753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4027549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4029339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4031144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4032942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4034736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4036516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4038359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4040212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4041991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4043798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4045590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4047447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4049266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4051067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4052869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4054740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4056523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4058732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4060541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4062479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4064340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4066156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4067959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4069779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4071654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4073521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 30%] 2024-08-08T21:00:09.4075332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4077144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4078945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4080742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4082561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4084375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4086209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4088078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4089888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4091695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4093490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4095332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4097193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4099002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4100791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4102605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4104416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4106249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4108064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4109900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4111747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4113697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4115370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4117186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4119027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4120895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4122723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4124470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4126273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4128103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4129889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4131677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4133494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4135364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4137182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4138983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4140797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4142646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4144498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4146313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4148106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4149914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4151702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4153541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4155364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4157191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4159442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4161338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4163142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4164940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4166777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4168636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4170424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4172252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4174031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4175822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4177608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4179426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4181198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4183034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4184894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4186727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4188510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4190350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4192218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4194034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4195838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4197636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4199450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4201257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4203060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4204855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4206697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4208550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4210343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4212125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4213963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4215782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4217621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4219414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4221213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4223035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4224875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4226626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4228427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4230306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4232108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4233911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4235708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4237527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4239364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4241228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4243050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4244851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4246641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4248446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4250235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4252051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4253915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4255743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4257548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4259772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4261599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4263476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4265363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4267166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4268961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4270753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4272566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4274373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4276167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4277953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4279824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4281696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4283486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4285303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4287144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4289017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4290802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4292614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4294399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4296210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4297986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4299780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4301601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4303436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4305261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4307069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4308859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4310710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4312548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4314326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4316143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4317946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4319729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4321513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4323336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4325148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4326984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4328842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4330664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4332476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4334319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4336157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4337964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4339768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4341545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4343332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4345115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4346932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4348710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4350536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4352374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4354237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4356023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4357823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4360069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4361985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4363758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4365566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4367354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4369170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4370949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4372732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4374604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4376476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4378265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4380052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4381854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4383686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4385518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4387292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4389194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4390926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4392687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4394474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4396275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4398106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4399925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4401717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4403509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4405329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4407142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4408990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4410788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4412620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4414403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4416209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4418007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4419834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4421615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4423501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4425319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4427151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4428907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4430729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4432583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4434390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4436183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4437964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4439764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4441586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4443351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4445129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4447004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4448874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4450722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4452503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4454424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4456301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4458117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4460318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4462152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4464004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4465803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4467622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4469418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4471351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4473216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4475035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4476858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4478778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4480611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4482422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4484230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4486065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4487890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4489693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4491512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4493340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4495197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4497052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4498890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4500719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4502596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4504469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4506301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4508132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4509942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4511757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4513573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4515391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4517195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4519036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4520928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4522739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4524534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4526397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4528251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4530089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4531889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4533715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4535522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4537349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4539141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4540957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4542822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4544736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4546520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4548343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4550200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4552094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4553937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4555749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4557563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4559772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4561600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4563405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4565234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4567138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4569022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4570817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4572632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4574511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4576370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4578167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4580016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4581823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4583662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4585436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4587255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4589099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4590944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4592822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4594642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4596491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4598346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4600219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4602013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4603839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4605627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4607439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4609245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4611176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4612868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4614722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4616574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4618399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4620215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4622020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4623908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4625792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4627597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4629408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4631259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4633069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4634889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4636704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4638575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4640466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4642271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4644101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4645894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4647739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4649567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4651387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4653240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4655089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4656878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4659073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4660923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4662842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4664713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4666541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4668350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4670186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4672034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4673910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4675722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4677592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4679377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4681185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4683015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4684838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4686687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4688558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4690372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4692180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4693969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4695807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4697675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4699485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4701302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4703095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4704918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4706740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4708546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4710401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4712276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4714099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4715896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4717721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4719589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4721480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4723285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4725121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4726941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4728785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4730596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4732429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4734267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4736141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4737930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4739751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4741567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4743448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4745305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4747116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4748941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4750761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4752570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4754420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4756254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4758116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4760334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4762146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4763973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4765792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4767683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4769571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4771399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4773229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4775022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4776840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4778635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4780451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4782307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4784183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4785985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4787814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4789702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4791455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4793307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4795139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4796928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4798745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4800547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4802382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4804167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4805965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4807832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4809703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4811520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4813323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4815187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4817054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4818864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4820671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4822479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4824278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4826068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4827849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4829667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4831529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4833374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4835170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4836969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4838836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4840701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4842508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4844314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4846138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4847921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4849728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4851538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4853415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4855256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4857128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4859318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4861187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4863069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4864986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4866778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4868588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4870385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4872181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4874001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4875807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4877611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4879466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4881355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4883170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4884976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4886823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4888689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4890503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4892305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4894100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4895925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4897735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4899526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4901344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4903192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4905076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 31%] 2024-08-08T21:00:09.4906870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4908686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4910526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4912387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4914286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4915971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4917768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4919600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4921385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4923204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4925009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4926883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4928708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4930503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4932329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4934154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4935996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4937844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4939671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4941513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4943314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4945122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4946957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4948777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4950630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4952484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4954351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4956156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4957959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4960312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4962233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4964065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4965861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4967670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4969467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4971299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4973094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4975029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4976847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4978683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4980474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4982288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4984135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4986024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4987822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4989642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4991451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4993273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4995080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4996884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.4998732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5001055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5002849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5004641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5006461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5008325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5010181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5011974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5013796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5015607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5017406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5019208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5021030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5022883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5024721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5026540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5028355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5030194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5032036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5033911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5035725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5037568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5039358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5041171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5042968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5044783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5046601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5048475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5050283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5052117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5053955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5055803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5057669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5059875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5061719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5063518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5065329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5067142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5068947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5070826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5072723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5074540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5076345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5078143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5080025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5081905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5083717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5085518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5087314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5089135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5090909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5092719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5094507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5096376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5098202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5100017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5101811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5103682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5105506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5107310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5109121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5110952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5112746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5114558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5116366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5118187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5120044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5121907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5123733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5125555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5127411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5129265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5131079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5132890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5134689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5136481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5138302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5140120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5141927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5143778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5145636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5147467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5149252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5151121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5152975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5154852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5156643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5158932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5160732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5162570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5164367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5166187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5168074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5169980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5171771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5173588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5175447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5177314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5179098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5180896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5182713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5184524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5186322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5188119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5189940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5191791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5193633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5195433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5197250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5199102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5200958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5202745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5204562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5206375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5208167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5209983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5211792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5213629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5215467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5217329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5219122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5220932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5222710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5224581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5226446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5228269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5230051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5231869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5233657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5235481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5237315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5239105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5240983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5242799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5244604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5246402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5248266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5250129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5251942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5253799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5255629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5257453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5259662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5261487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5263389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5265274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5267079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5268873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5270679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5272571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5274426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5276240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5278051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5279883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5281668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5283488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5285291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5287158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5288983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5290797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5292611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5294444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5296280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5298144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5299951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5301768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5303580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5305379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5307188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5308986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5310816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5312645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5314457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5316271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5318067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5319919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5321791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5323590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5325394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5327188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5329005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5330815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5332596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5334405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5336254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5338131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5339929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5341753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5343601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5345473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5347264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5349079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5350869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5352685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5354514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5356316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5358118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5360400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5362276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5364076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5365883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5367755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5369621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5371415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5373234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5375044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5376840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5378631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5380459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5382273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5384116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5385964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5387785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5389601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5391464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5393319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5395107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5396933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5398701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5400494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5402284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5404111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5405896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5407746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5409604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5411427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5413205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5415056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5416897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5418722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5420628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5422307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5424125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5425933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5427735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5429534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5431385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5433245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5435042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5436828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5438635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5440466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5442294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5444071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5445880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5447683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5449472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5451261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5453054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5454959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5456781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5458973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5460851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5462681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5464560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5466458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5468275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5470116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5471905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5473719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5475531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5477367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5479217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5481097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5482887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5484700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5486492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5488324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5490182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5491994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5493807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5495603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5497403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5499213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5501010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5502797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5504674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5506552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5508367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5510175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5512051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5513931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5515737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5517571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5519394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5521231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5523030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5524853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5526651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5528520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5530354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5532177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5533985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5535864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5537697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5539523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5541317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5543153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5544954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5546766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5548593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5550416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5552267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5554166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5556004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5557842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5560134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5562045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5563875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5565707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5567515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5569323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5571145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5572956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5574775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5576618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5578503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5580341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5582133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5584004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5585871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5587709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5589510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5591332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5593132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5594960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5596747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5598560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5600416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5602331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5604132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5605958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5607811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5609677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5611484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5613291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5615100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5616913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5618694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5620490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5622305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5624153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5625997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5627801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5629605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5631455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5633297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5635095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5636928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5638752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5640540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5642360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5644178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5646019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5647861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5649739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5651557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5653399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5655283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5657158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5659347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5661212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5663002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5664818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5666632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5668477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5670269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5672163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5674074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5675885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5677714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5679583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5681476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5683301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5685121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5686929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5688765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5690599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5692426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5694241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5696122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5698002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5699833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5701632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5703478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5705341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5707128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5708943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5710760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5712598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5714387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5716213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5718019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5719898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5721746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5723578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5725382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5727261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5729098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5730916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5732727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5734554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5736360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 32%] 2024-08-08T21:00:09.5738168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5739986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5741815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5743662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5745520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5747339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5749148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5750949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5752784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5754692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5756507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5758311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5760521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5762347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5764161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5765949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5767846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5769748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5771583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5773383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5775206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5777093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5779005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5780806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5782641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5784454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5786297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5788096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5789919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5791752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5793625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5795409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5797209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5799039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5800897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5802755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5804562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5806383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5808216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5810012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5811822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5813647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5815500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5817354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5819150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5820976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5822805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5824649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5826500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5828308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5830162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5831939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5833763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5835561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5837393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5839210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5841070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5842870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5844696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5846479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5848335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5850182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5852005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5853829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5855648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5857453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5859648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5861491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5863398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5865307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5867130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5868952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5870763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5872660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5874545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5876365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5878174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5879993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5881795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5883597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5885396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5887267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5889130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5890927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5892744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5894548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5896430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5898263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5900083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5901894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5903730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5905519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5907338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5909154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5911025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5912864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5914695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5916513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5918355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5920195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5922047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5923853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5925664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5927460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5929251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5931076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5932892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5934702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5936535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5938401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5940218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5942020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5943861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5945736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5947549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5949359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5951145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5952948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5954822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5956613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5958848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5960700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5962599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5964398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5966211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5968063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5969942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5971720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5973522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5975315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5977134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5978912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5980725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5982521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5984397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5986245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5988042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.5998381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6000474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6002348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6004176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6005993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6007832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6009627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6011454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6013262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6015099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6016937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6018797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6020606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6022407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6024255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6026099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6027921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6029737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6031541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6033340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6035155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6036974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6038773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6040608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6042473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6044283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6046088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6047930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6049786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6051624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6053418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6055309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6057122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6059440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6061281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6063109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6065008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6066910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6068695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6070509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6072315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6074200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6076056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6077894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6079672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6081482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6083280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6085081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6086896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6088786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6090648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6092552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6094277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6096101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6097950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6099808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6101635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6103472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6105261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6107062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6108869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6110675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6112496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6114354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6116159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6117986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6119780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6121617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6123458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6125291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6127081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6128888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6130682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6132513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6134295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6136140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6137984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6139917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6141600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6143396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6145259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6147113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6148918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6150717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6152553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6154408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6156205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6157986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6160270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6162180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6163983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6165775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6167585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6169478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6171367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6173155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6174972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6176792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6178583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6180398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6182205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6184033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6185867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6187739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6189552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6191384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6193217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6195086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6196873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6198692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6200473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6202285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6204084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6205896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6207696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6209532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6211387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6213200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6215001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6216837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6219083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6220893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6222701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6224498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6226323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6228143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6229956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6231760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6233624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6235490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6237283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6239095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6241018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6242793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6244565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6246375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6248178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6250000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6251787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6253597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6255447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6257318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6259510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6261355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6263155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6265090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6266951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6268741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6270561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6272368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6274167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6275966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6277788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6279607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6281466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6283327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6285132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6286929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6288765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6290594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6292417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6294229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6297747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6301325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6304705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6308185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6311784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6315238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6318779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6322268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6325748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6329264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6332885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6336298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6339797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6343298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6346790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6350302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6353848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6357319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6361832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6365430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6368902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6372367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6375857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6379429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6382965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6386441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6389929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6393430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6396917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6400394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6403872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6407393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6410917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6414376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6417863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6421365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6424936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6428452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6431947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6435452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6438924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6442389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6445856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6449329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6452870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6456365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6460300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6463819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6467295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6470879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6474510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6477914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6481398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6484871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6488347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6491838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6495321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6498832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6502392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6505895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6509376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6512840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6516381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6519913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6523381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6526844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6530301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6533786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6537248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6540691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6544156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6547689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6551209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6554732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6558206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6562236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6565801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6569246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6572732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6576222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6579686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6583144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6586621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6590131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6593664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6597214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6600712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6604211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6607749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6611268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6614742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6618208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6621672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6625123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6628592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6632082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6635549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6639037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6642574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6646053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6649502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6653049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6656572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6660499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6663963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6667410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6670887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6674376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6677857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6681325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6684878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6688458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6691933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6695476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6698916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6702449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6705910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6709331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6712784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6716270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6719726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6723190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6726645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6730178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6733673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6737120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6740589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6744083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6747707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6751127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6754648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6758173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6762052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6765545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6769044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6772570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6776127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6779669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6783145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6786614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6790064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6793573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6797120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6800608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6804079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6807538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6810999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6814490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6817960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6821478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6824980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6828462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6831923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6835367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 33%] 2024-08-08T21:00:09.6838889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6842424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6845899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6849353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6852879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6856371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6860198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6863688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6867238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6870771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6874227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6877667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6881135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6884688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6888227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6891693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6895154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6898637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6902110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6905559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6909034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6912518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6916019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6919518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6922997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6926498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6930026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6933545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6937038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6940524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6944007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6947477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6950948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6954471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6957932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6961913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6965438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6968920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6972402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6975921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6979469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6982951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6986419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6989848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6993320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.6996803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7000265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7003718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7007230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7010758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7014225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7017682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7021150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7024674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7028188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7031641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7035088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7038545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7041994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7045427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7048874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7052429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7055924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7059762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7063258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7066733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7070272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7073790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7077262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7080746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7084212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7087675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7091157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7094653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7098194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7101725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7105187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7108687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7112168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7115692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7119214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7122695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7126138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7129584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7133047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7136543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7140008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7143464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7146986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7150547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7154065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7157531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7161583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7165086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7168553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7172004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7175474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7178974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7182448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7185923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7189405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7192974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7196505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7199970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7203443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7206967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7210474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7213916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7217387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7220853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7224325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7227785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7231258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7234739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7238254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7241746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7245219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7248693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7252258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7255775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7259674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7263212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7266676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7270137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7273613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7277103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7280567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7284115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7287633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7291096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7294547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7297976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7301491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7305028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7308489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7311929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7315387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7318871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7322338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7325779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7329285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7332843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7336296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7339769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7343251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7346798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7350327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7353850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7357330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7361252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7364753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7368230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7371694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7375275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7378805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7382240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7385706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7389208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7392749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7396286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7399753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7403229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7406696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7410154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7413621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7417094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7420551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7424057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7427562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7431054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7434530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7438038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7441563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7445044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7448511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7451992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7455457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7459304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7462792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7466231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7469787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7473314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7476775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7480226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7483740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7487282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7490745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7494191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7497633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7501106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7504574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7508027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7511503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7515054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7518562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7522026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7525502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7529000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7532515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7536038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7539504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7542958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7546401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7549831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7553337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7556813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7560765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7564302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7567743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7571211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7574665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7578163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7581677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7585153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7588594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7592042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7595506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7598994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7602460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7605962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7609486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7612948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7616410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7619866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7623371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7626884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7630325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7633740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7637197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7640663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7644113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7647554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7651010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7654573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7658060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7661906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7665373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7668933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7672481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7675910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7679356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7682831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7686302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7689751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7693214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7696678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7700203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7703703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7707160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7710612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7714090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7717563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7721009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7724467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7727900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7731334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7734773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7738239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7741688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7745171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7748666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7752183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7755656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7759548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7763046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7766619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7770089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7773540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7777014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7780509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7783985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7787443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7790959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7794469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7797912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7801341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7804794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7808320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7811823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7815263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7818698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7822165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7825625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7829071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7832554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7836072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7839609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7843120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7846625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7850143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7853778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7857327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7861357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7864876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7868375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7871869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7875355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7878859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7882338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7885944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7889503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7893018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7896523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7900067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7903627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7907134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7910624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7914078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7917579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7921091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7924578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7928061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7931606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7935154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7938656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7942155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7945707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7949267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7952812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7956318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7960184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7963718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7967183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7970646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7974132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7977740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7981298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7984752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7988239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7991810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7995385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.7998851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8002346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8005820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8009290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8012755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8016237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8019754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8023292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8026853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8030332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8033839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8037372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8040891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8044376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8047857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8051321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8053145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8054953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8056862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8058975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8060875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8062749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8064580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8066355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8068223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8070089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8071920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8073715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8075545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8077358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8079205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8081008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8082836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8084687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8086636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8088382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8090193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8092006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8093849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8095701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8097506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8099333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8101149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8102960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8104762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8106579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8108430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8110286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8112086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8113912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8115733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8117557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8119420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8121234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8123082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8124879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8126717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8128532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8130373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8132228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8134103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8135897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8137748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8139499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8141357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8143212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8145048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8146844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8148645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8150470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8152333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8154153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8156002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8157869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8160099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8161939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8163735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8165657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8167541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8169348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8171155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8172977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8174791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8176625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8178399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8180247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8182130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8183913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8185721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8187523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8189391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8191220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8193022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8194821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8196638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8198414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8200226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8202032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8203871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8205698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8207547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8209374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8211196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8213061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8215497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8217332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8219163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8220975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8222777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8224585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8226414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8228185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8230027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8231903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8233716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8235521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8237360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8239207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8241042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8242832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8244649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8246450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8248275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 34%] 2024-08-08T21:00:09.8250057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8251873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8253760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8255645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8257429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8259630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8261555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8263464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8265253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8267080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8268878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8270672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8272468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8274254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8276068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8277936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8279806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8281603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8283422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8285272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8287116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8288911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8290734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8292545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8294349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8296147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8297974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8299792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8301617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8303488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8305304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8307136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8308990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8310847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8312643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8314460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8316241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8318040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8319843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8321678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8323457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8325307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8327152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8328983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8330768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8332562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8334425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8336341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8338145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8339941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8341764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8343589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8345406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8347208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8349077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8350938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8352798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8354600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8356419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8358258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8360528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8362319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8364127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8365957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8367745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8369568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8371360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8373275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8375125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8377017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8378856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8380675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8382539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8384414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8386209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8388040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8389823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8391618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8393434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8395255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8397101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8398954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8400756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8402556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8404345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8406165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8408026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8409825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8411623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8413408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8415211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8417014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8418804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8420593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8422444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8424323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8426110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8427926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8429775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8431658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8433448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8435271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8437074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8438904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8440693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8442506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8444298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8446166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8447994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8449802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8451595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8453504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8455351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8457142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8459443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8461299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8463104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8464899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8466724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8468541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8470475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8472362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8474192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8476009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8477887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8479754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8481581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8483403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8485205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8487026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8488820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8490652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8492432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8494307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8496161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8497999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8499787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8501603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8503440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8505318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8507103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8508924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8510729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8512555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8514345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8516142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8518005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8519870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8521680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8523487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8525316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8527174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8529028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8530826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8532643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8534447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8536241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8538036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8539855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8541709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8543547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8545360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8547155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8548986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8550836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8552755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8554560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8556392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8558172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8560453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8562276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8564106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8565985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8567875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8569675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8571504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8573294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8575149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8577053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8578848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8580642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8582425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8584229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8586038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8587837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8589618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8591474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8593327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8595123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8596904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8598768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8600623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8602423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8604221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8606023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8607852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8609642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8611460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8613269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8615142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8616979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8618799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8620594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8622461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8624283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8626090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8627897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8629724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8631504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8633309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8635110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8636919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8638765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8640607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8642433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8644241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8646089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8647928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8649762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8651572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8653421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8655221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8657043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8659296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8661151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8663046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8664934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8666738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8668524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8670412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8672283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8674110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8675896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8677707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8679503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8681322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8683112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8684926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8686764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8688639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8690422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8692236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8694034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8695889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8697738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8699536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8701358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8703163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8704961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8706759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8708570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8710409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8712251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8714030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8715839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8717645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8719543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8721380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8723193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8725000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8726789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8728592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8730402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8732230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8734066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8735924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8737728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8739566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8741356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8743221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8745069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8746909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8748705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8750522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8752365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8754202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8755987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8757815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8760159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8762004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8763808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8765674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8767538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8769413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8771206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8773007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8774818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8776615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8778414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8780200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8782012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8783878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8785749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8787543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8789345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8791227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8793067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8794874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8796667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8798481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8800260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8802064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8803858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8805676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8807493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8809351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8811140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8812960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8814785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8816619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8818429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8820238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8822036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8823818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8825643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8827464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8829272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8831112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8832980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8834792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8836596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8838431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8840282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8842075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8843858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8845641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8847431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8849256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8851043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8852890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8854758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8856636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8859043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8861031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8862839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8864825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8866688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8868493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8870289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8872119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8873901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8875722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8877521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8879523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8881335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8883186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8884988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8887214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8889072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8890942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8892739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8894551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8896346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8898127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8899940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8901746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8903581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8905411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8907220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8909018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8910801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8912622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8914478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8916280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8918085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8919871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8921664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8923487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8925262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8927065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8928894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8930753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8932530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8934322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8936156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8938017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8939784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8941582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8943380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8945184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8946947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8948745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8950542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8952446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8954297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8956086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8957898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8960038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8961935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8963727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8965730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8967574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8969417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8971213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8973030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8974821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8976695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8978599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8980426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8982237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8984108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8985866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8987708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8989527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8991307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8993115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8994908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8996737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.8998517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9000360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9002209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9004040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9005848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9007684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9009539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9011341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9013123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9014920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9016724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9018520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9020316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9022098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9023947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9025803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9027596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9029383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9031188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9033043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9034883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9036717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9038483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9040288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9042068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9043872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9045738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9047550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9049380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9051190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9053032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9054861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9056695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9058901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9060636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9062455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9064224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9066017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9067811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9069638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9071420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9073302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9075206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9077008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9078805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 35%] 2024-08-08T21:00:09.9080655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9082532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9084403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9086141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9087947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9089741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9091542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9093340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9095193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9096993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9098848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9100638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9102423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9104249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9106107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9107879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9109674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9111465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9113287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9115064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9116862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9118648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9120504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9122328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9124126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9125919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9127747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9129563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9131395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9133215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9135033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9136846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9138677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9140440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9142304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9144089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9145934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9147754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9149545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9151336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9153211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9155072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9156881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9158980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9160725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9162511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9164439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9166115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9168002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9169865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9171733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9173500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9175309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9177168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9179056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9180842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9182662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9184461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9186294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9188089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9189881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9191683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9193519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9195356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9197189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9198960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9200799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9202653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9204414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9206228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9208036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9209830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9211625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9213435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9215235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9217068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9218903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9220703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9222529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9224351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9226213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9228003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9229826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9231603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9233403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9235184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9236998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9238759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9240586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9242422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9244241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9246012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9247857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9249710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9251506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9253333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9255129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9256962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9258960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9260789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9262576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9264475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9266359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9268165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9269962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9271772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9273634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9275499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9277299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9279097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9280913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9282685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9284486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9286277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9288142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9289964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9291768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9293570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9295378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9297193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9299037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9300830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9302652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9304424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9306226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9308029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9309834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9311669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9313507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9315310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9317116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9318916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9320740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9322581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9324370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9326166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9327939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9329748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9331655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9333441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9335613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9337508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9339363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9341152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9342947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9344798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9346643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9348411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9350294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9352053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9353880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9355655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9357465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9359509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9361498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9363365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9365169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9366946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9368818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9370638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9372433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9374211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9376013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9377907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9379585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9381395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9383191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9396990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9399008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9400837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9402644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9404415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9406243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9408082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9409912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9411703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9413516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9415315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9417145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9418931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9420773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9422610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9424420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9426195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9427990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9429823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9431689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9433463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9435251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9437058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9438860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9440654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9442435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9444230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9446069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9447901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9449754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9451486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9453402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9455241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9457023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9459123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9460958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9462743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9464546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9466327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9468139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9470009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9471880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9473666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9475481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9477346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9479208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9480992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9482873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9484576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9486374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9488164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9490009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9491777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9493608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9495475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9497292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9499090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9500882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9502733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9504583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9506385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9508182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9509996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9511778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9513571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9515354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9517206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9519058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9520835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9522636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9524432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9526294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9528118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9529923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9531747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9533587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9535382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9537212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9539037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9540930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9542768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9544597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9546409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9548249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9550098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9552006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9553830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9555647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9557455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9559513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9561369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9563213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9565085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9566981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9568804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9570623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9572435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9574300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9576185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9578000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9579820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9581626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9583541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9585294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9587096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9588966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9590830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9592681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9594485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9596315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9598165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9600041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9601826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9603651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9605454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9607290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9609082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9610905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9612718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9614576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9616434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9618241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9620063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9621917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9623768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9625568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9627400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9629221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9631036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9632835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9634653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9636476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9638332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9640183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9642002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9643824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9645642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9647497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9649296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9651129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9652981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9654784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9656630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9658789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9660501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9662404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9664293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9666208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9667937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9669878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9671712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9673665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9675459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9677175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9679069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9680871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9682666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9684480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9686340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9688252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9690012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9691810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9693677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9695543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9697367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9699155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9700981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9702850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9704657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9706472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9708227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9710139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9711969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9713785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9715703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9717496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9719342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9721171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9722980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9724829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9726656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9728463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9730270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9732100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9733973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9735842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9737602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9739420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9741249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9743076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9744970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9746847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9748605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9750408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9752268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9754084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9755897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9757752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9759899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9761739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9763555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9765364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9767266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9769160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9770961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9772792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9774680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9776409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9778191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9780003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9781863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9783763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9785549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9787364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9789169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9791046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9792878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9794696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9796519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9798360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9800156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9801966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9803807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9805678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9807549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9809364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9811201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9813027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9814896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9816755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9818569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9820386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9822201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9823994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9825820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9827636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9829489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9831352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9833147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9834997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9836790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9838673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9840528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9842357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9844150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9845968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9847785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9849626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9851418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9853281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9855141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9857033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9859110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9860972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9862857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9864732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9866534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9868327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9870145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9871956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9873762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9875560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9877382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9879300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9881179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9882973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9884791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9886646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9888504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9890302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9892120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9893956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9895746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9897563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9899379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9901220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9903038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9904906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9906703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9908523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9910342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9912196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9913992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9915821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9917608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9919428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9921232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 36%] 2024-08-08T21:00:09.9923054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9924865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9926698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9928582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9930398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9932264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9934142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9935944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9937777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9939608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9941414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9943236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9945050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9946864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9948670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9950527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9952420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9954204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9956018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9957818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9959899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9961770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9963586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9965393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9967229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9969017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9970830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9972628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9974508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9976355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9978164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9979965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9981794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9983642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9986182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9988002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9989820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9991626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9993430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9995249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9997052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:09.9998889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0000720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0002529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0004341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0006139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0007968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0009827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0011632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0013430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0015249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0017028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0018860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0020652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0022545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0024420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0026263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0028061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0029880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0031737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0033625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0035431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0037243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0039045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0040870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0042654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0044464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0046262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0048116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0049971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0051773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0053638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0055501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0057347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0059621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0061493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0063307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0065134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0066916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0068741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0070564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0072475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0074351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0076161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0078021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0079867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0081761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0083558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0085384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0087163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0088973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0090779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0092615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0094399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0096258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0098112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0099940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0101726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0103592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0105487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0107294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0109094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0110892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0112721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0114543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0116363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0118168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0120033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0121903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0123717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0125519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0127370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0129221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0131020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0132810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0134633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0136448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0138233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0140056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0141865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0143740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0145571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0147381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0149177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0150999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0152873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0154739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0156540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0158703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0160422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0162235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0164038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0165885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0167756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0169605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0171418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0173212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0174997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0176838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0178712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0180523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0182325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0184114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0185923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0187732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0189532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0191359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0193261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0195053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0196863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0198664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0200532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0202424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0204219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0206046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0207855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0209687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0211483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0213305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0215096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0216953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0218785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0220594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0222393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0224266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0226093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0227894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0229715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0231529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0233329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0235118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0236930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0238740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0240588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0242435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0244261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0246080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0247940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0249841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0251658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0253519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0255335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0257139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0259147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0260980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0262759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0264634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0266504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0268331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0270117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0271995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0273863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0275693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0277475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0279282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0281079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0282995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0284690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0286501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0288344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0290223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0292036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0293834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0295691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0297546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0299348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0301141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0302946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0304739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0306531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0308321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0310134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0311981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0313831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0315620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0317429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0319237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0321053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0322907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0324711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0326542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0328328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0330143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0331963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0333802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0335632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0337502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0339304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0341152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0342929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0344790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0346626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0348443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0350223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0352014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0353886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0355688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0357507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0359570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0361480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0363299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0365101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0366890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0368769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0370628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0372427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0374219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0376035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0377845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0379639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0381454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0383313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0385188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0386973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0388785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0390577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0392435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0394255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0396057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0397854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0399681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0401456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0403256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0405043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0406867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0408685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0410520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0412333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0414145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0415987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0417822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0419643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0421465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0423271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0425064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0426880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0428696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0430507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0432344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0434211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0436009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0437803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0439631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0441468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0443299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0445081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0446892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0448684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0450510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0452336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0454150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0455994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0457866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0459851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0461676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0463483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0465385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0467256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0469053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0470878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0472696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0474500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0476303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0478113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0479967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0481826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0483618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0485434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0487253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0489079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0490919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0492729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0494538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0496383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0498129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0499926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0501755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0503569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0505409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0507209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0509032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0510815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0512668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0514509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0516339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0518116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0519921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0521703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0523525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0525301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0527094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0528932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0530783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0532574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0534362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0536214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0538063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0539855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0541638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0543452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0545282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0547059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0548854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0550675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0552570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0554433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0556230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0558035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0560127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0561997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0563817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0565612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0567425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0569202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0571006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0572807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0574640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0576475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0578354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0580146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0581972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0583889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0585651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0587463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0589283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0591102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0592909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0594750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0596586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0598401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0600257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0602129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0603955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0605772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0607627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0609495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0611299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0613104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0614903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0616726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0618536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0620333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0622152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0624001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0625882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0627676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0629498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0631302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0633219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0635064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0636888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0638712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0640558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0642358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0644188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0645997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0647882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0649751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0651569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0653429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0655245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0657087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0659186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0661036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0662866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0664682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0666487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0668311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0670131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0672016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0673887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0675713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0677528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0679336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0681209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0683094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0684930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0686737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0688560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0690369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0692209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0694005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0695866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0697709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0699529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0701312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0703123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0704969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0706845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0708629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0710453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0712260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0714080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0715889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0717684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0719556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0721420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0723225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0725031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0726868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0728735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0730603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0732429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0734247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0736074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0737886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0739705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0741528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0743334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0745169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0747039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0748854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0750694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 37%] 2024-08-08T21:00:10.0752580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0754449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0756256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0758094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0760107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0761937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0763748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0765588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0767379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0769308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0771161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0772987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0774803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0776669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0778574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0780421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0782226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0784041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0785861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0787674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0789483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0791284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0793148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0795015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0796825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0798620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0800492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0802357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0804147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0805972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0807777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0809601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0811387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0813214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0815037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0816912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0818754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0820574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0822376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0824255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0826104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0827928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0829728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0831557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0833337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0835128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0836946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0838756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0840606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0842463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0844281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0846098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0847947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0849787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0851614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0853479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0855296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0857102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0859154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0860997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0862813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0864700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0866585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0868433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0870236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0872123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0873993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0875814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0877600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0879418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0881230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0883062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0884853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0886674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0888520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0890404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0892198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0894012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0895814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0897667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0899517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0901317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0903152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0904978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0906784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0908588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0910427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0912280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0914139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0915940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0917759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0919557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0921436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0923246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0925067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0926885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0928674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0930507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0932314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0934141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0935968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0937825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0939629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0941456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0943247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0945104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0946959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0948798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0950599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0952467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0954273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0956119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0957911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0960042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0961953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0963757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0965581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0967347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0969236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0971112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0972913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0974714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0976540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0978354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0980160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0981954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0983766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0985616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0987473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0989268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0991082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0992966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0994811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0996627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.0998453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1000273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1002063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1003884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1005677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1007495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1009314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1011252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1013052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1014886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1016715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1018568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1020387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1022204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1024014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1025805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1027617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1029428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1031236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1033070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1034943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1036758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1038567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1040405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1042271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1044082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1045885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1047684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1049479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1051290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1053116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1054929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1056766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1058892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1060701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1062505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1064303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1066203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1068046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1069855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1071667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1073501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1075288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1077115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1078954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1080820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1082704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1084513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1086341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1088161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1090011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1091865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1093680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1095489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1097292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1099085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1100909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1102725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1104607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1106439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1108270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1110084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1111869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1113731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1115579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1117411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1119204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1121027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1122835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1124678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1126472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1128329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1130181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1132015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1133819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1135646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1137492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1139357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1141139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1142931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1144748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1146562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1148366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1150165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1152026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1153933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1155795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1157584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1160185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1162168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1164055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1165882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1167695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1169522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1171321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1173147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1174956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1176797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1178653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1180537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1182338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1184161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1185997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1187851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1189652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1191482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1193270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1195095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1196887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1198723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1200514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1202348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1204977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1206793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1208603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1210445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1212320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1214138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1215944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1217742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1219564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1221375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1223185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1224987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1226843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1228691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1230495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1232285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1234125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1235995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1237791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1239601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1241394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1243216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1244994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1246807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1248611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1250497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1252385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1254215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1256031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1257872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1261670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1263650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1265449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1267274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1269087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1270895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1272715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1274519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1276407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1278329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1280108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1283755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1287231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1290808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1294312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1297846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1301396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1304894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1308383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1311877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1315377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1318902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1322446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1325979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1329470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1332949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1336481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1340038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1343528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1347015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1350514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1354139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1357509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1361235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1364796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1368372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1371850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1375326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1378806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1382356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1385930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1389398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1392879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1396376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1399849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1403320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1406791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1410298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1413830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1417366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1420854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1424361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1427875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1431416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1434895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1438381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1441856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1445303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1448776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1452255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1455782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1459533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1463120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1466593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1470064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1473581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1477137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1480640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1484122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1487602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1491109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1494635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1498111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1501600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1505195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1508769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1512261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1515747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1519284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1522814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1526286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1529750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1533226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1536727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1540220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1543687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1547192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1550747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1554332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1557807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1561553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1565157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1568699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1572168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1575656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1579173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1582664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1586139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1589645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1593138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1596701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1600267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1603748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1607243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1610699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1614190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1617723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1621217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1624701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1628170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1631642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1635137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1638596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1642124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1645638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1649134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1652661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1656128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1659976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1663559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1667057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1670538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1674028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1677538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1681031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1684489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1688042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1691590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1695049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1698504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1701976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1705518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1709022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1712489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1715975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1719467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1722936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1726400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1729863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1733356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1736881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1740425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1743909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1747418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1750955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1754511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1758001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1761741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1765246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1768725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1772202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1775671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1779124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1782664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1786217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1789717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1793189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1796755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1800260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1803760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1807230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1810693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1814170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1817654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1821123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1824573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1828121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1831679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1835144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1838627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1842150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1845675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1849153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1852662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1856143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1859973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1863375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1866829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 38%] 2024-08-08T21:00:10.1870282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1873856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1877396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1880857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1884328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1887813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1891337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1894853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1898331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1901832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1905337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1908787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1912280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1915776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1919337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1922869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1926355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1929864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1933355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1936888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1940404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1943885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1947352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1950805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1954333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1957839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1961529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1964995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1968544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1972113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1975578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1979047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1982604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1986149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1989624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1993089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.1996563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2000060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2003536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2006998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2010460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2013994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2017528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2020990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2024479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2027973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2031479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2034916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2038372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2041850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2045322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2048776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2052250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2055776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2059531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2063063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2066529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2070020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2073557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2077073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2080543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2084052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2087534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2091008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2094483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2097973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2101435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2104946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2108470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2111927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2115381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2118826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2122334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2125846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2129302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2132756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2136215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2139696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2143159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2146594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2150116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2153680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2157154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2160846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2164313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2167881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2171413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2174859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2178336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2181823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2185315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2188754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2192203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2195647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2199151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2202655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2206112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2209577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2213079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2216619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2220075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2223562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2227020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2230466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2233932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2237405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2240873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2244381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2247900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2251394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2254918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2258799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2262283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2265787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2269267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2272726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2276207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2279677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2283131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2286568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2290070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2293626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2297085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2300547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2304018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2307511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2311031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2314481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2317939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2321418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2324884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2328345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2331803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2335349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2338876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2342342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2345825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2349313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2352865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2356388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2360084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2363572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2367033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2370463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2373931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2377408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2380946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2384465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2387926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2391401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2394859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2398375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2401890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2405367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2408827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2412266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2415732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2419217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2422669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2426144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2429662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2433221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2436695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2440143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2443648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2447131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2450579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2454048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2457508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2461261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2464710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2468139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2471597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2475141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2478697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2482108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2485568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2489135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2492659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2496114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2499596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2503101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2506593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2510070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2513539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2517041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2520581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2524107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2527576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2531043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2534498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2537967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2541493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2544980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2548454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2551917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2555427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2559136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2562604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2566133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2569661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2573137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2576595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2580031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2583662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2587105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2590571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2594035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2597504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2600979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2604457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2607898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2611351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2614878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2618377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2621812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2625267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2628764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2632271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2635711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2639170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2642637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2646097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2649536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2653020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2656503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2660270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2663795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2667277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2670769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2674286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2677815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2681280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2684755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2688227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2691679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2695128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2698575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2702016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2705495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2709006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2712480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2715935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2719370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2722869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2726380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2729836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2733270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2736721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2740198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2743647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2747090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2750605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2754184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2757659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2761392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2764869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2768436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2771995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2775450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2778905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2782371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2785809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2789228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2792690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2796158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2810851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2814652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2818391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2822151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2825906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2829647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2833317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2837052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2840768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2844446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2848196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2851898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2855658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2859747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2863511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2867214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2870944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2874727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2878520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2882231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2885945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2889637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2893319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2897066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2900752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2904442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2908203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2911983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2915699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2919407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2923135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2926911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2930651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2934368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2938097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2941823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2945537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2949282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2953067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2956851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2960912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2964618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2968332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2972045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2975808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2979569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2983298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2987061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2990781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2994480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.2998181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3001881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3005612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3009346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3013166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3016958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3020737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3024551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3028333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3032129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3035900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3039674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3043451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3047195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3050969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3054776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3058842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3062765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3064666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3066596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3068629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3070717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3072533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3074488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3076433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3078410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3080308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3082263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3084181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3086184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3088162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3090151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3092090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3094113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3096092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3098053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3099996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3101944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3103871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3105812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3107769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3109717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3111739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3113711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3115674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3117596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3119557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3121528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3123536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3125491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3127424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3129360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3131310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3133270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3135197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3137193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3139187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3141135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3143055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3145035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3147020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3149037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3150959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3152964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3154907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3156860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3158953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3160923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3162921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3164929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3166864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3168771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3170829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3172702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3174704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3176633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3178578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3180540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3182477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3184398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3186371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3188381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3190385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3192327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3194299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3196260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3198236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3200220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3202132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3204095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3206012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3207963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3209893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3211870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3213803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3215825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3217816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3219789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3221725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3223721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3225706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3227672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3229583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3231530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3233492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3235461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3237459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3239362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3241339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3243333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3245287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3247225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3249217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3251202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3253180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3255112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3257072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3259197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3261148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3263068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3265013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3267034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3269028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3270961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3272897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3274887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3276841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3278765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3280706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3282654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3284593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3286556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3288501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3290513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3292449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3294474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3296364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 39%] 2024-08-08T21:00:10.3298275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3300225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3303023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3304945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3306863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3308809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3310740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3312669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3314593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3316526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3318498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3320507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3322455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3324403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3326354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3328343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3330368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3332299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3334229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3336148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3338146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3340038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3341995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3343969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3345949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3347849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3349793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3351733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3353793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3355772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3357706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3359916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3361891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3363800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3365759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3367708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3369708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3371714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3373647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3375609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3377562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3379576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3381550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3383506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3385457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3387404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3389319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3391270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3393217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3395129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3397089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3399048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3401034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3402905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3404892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3406873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3408802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3410691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3412613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3414546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3416512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3418429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3420352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3422346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3424366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3426294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3428233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3430206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3432199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3434118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3436038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3437988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3439927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3441860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3443767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3445722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3447711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3449679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3451612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3453610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3455612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3457587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3459768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3461727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3463663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3465581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3467518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3469467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3471444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3473433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3475443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3477368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3479341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3481260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3483290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3485283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3487242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3489163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3491089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3493037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3494970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3496914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3498863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3500874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3502821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3504755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3506690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3508672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3510711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3512641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3514580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3516535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3518477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3520413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3522354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3524331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3526316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3528234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3530187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3532117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3534091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3536042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3537973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3539906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3541881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3543799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3545751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3547683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3549646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3551579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3553604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3555565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3557520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3559798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3561823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3563758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3565691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3567629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3569567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3571497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3573421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3575368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3577375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3579364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3581299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3583242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3585182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3587163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3589132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3591076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3593003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3594934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3596887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3598791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3600745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3602739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3604753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3606662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3608617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3610608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3612629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3614550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3616505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3618448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3620397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3622334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3624248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3626194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3628169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3630160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3632051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3634004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3635950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3637924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3639897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3641838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3643792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3645761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3647645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3649563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3651502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3653496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3655495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3657449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3659682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3661641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3663678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3665700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3667680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3669618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3671584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3673520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3675492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3677381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3679403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3681381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3683333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3685281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3687218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3689218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3691208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3693146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3695086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3697034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3698971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3700881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3702801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3704735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3706717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3708702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3710699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3712574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3714570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3716533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3718476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3720396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3722348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3724263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3726194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3728104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3730071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3731951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3733834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3735647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3737476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3739294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3741148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3742949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3744773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3746575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3748372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3750201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3752026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3753887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3755743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3757611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3759664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3761490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3763294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3765181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3767052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3768852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3770644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3772461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3774300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3776063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3777866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3779730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3781617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3783399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3785209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3787016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3788905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3790747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3792546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3794358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3796187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3797971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3799795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3801601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3803467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3805317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3807118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3808931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3810728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3812570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3814406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3816220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3818030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3819829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3821614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3823435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3825249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3827081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3828918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3830730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3832547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3834333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3836182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3838035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3839869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3841662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3843469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3845265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3847093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3848875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3850692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3852521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3854431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3856220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3858020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3860152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3862070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3863849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3865639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3867449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3869260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3871052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3872844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3874667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3876549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3878441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3880238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3882065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3883927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3885782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3887578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3889398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3891208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3892998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3894816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3896620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3898442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3900264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3902124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3903925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3905749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3907529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3909431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3911291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3913101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3914886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3916694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3918491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3920328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3922107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3923934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3925801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3927622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3929417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3931213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3933078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3934931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3936735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3938527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3940338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3942136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3943939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3945715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3947564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3949410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3951209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3953002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3954841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3956710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3958832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3960673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3962481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3964310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3966094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3967902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3969699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3971531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3973436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3975486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3977289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3979123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3980978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3982845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3984644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3986441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3988232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3990009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3991816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3993626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3995421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3997246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.3999105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4000907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4002697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4004522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4006373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4008177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4009974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4011782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4013562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4015389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4017178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4018990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4020851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4022734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4024523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4026327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4028117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4029971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4031779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4033573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4035368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4037189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4038966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4040754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4042564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4044406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4046246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4048036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4049839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4051638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4053520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4055374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4057185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4059264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4061084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4062866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4064674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4066482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4068365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4070218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4071994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4073801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4075578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4077420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4079269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4081090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4082866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4084665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4086452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4088269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4090036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4091827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4093663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4095536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4097314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4099092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4101000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4102876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4104675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4106471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4108283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4110092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4111893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4113688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4115494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4117327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4119170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4120948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4122756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4124604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4126439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4128221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4130013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4131841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4133615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4135417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4137212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4139031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4140845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4142692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4144495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4146317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4148099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4149962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4151804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4153672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4155471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 40%] 2024-08-08T21:00:10.4157261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4159288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4161106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4162890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4164733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4166607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4168409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4170216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4172001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4173870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4175742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4177535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4179324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4181137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4182946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4184739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4186526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4188324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4190190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4192039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4193827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4195627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4197494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4199330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4201135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4202923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4204739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4206504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4208307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4210094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4211915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4213727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4215562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4217371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4219209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4221007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4222826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4224622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4226420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4228206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4229983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4231791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4233603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4235394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4237217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4239086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4240887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4242684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4244469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4246299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4248153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4249921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4251710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4253532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4255349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4257120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4259162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4261038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4262948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4264693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4266480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4268269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4270162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4272014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4273803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4275621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4277435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4279237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4281040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4282849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4284698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4286602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4288378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4290180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4291972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4293800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4295618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4297432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4299238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4301031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4302813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4304597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4306417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4308193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4310040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4311880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4313699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4315472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4317315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4319268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4320996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4322780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4324583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4326379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4328207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4329999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4331812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4333652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4335501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4337287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4339060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4340909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4342759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4344545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4346328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4348137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4349939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4351728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4353552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4355352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4357191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4359241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4361033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4362824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4364642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4366479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4368347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4370132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4371953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4373736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4375534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4377317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4379126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4380944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4382791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4384569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4386384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4388159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4389986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4391827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4393618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4395415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4397192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4399002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4400804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4402602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4404386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4406244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4408102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4409900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4411693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4413547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4415415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4417220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4419014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4420798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4422609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4424378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4426170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4427961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4429819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4431646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4433446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4435234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4437103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4438924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4440718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4442498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4444312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4446082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4447877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4449677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4451482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4453363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4455208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4457013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4459021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4460830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4462687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4464555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4466345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4468138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4469908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4471707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4473506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4475297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4477128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4478976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4480803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4482655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4484335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4486154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4488015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4489781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4491576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4493365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4495182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4496959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4498766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4500596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4502460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4504232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4506016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4507815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4509659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4511492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4513264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4515069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4516862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4518651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4520422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4522217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4524029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4525845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4527667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4529465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4531265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4533096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4534911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4536699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4538520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4540289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4542093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4543879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4545702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4547477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4549326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4551852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4553705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4555470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4557258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4559375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4561281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4563049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4564835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4566635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4568429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4570219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4571992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4573845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4575718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4577487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4579250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4581047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4582887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4584723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4586494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4588299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4590098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4591898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4593676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4595452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4597243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4599042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4600887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4602672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4604485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4606288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4608129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4609913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4611729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4613497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4615281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4617058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4618881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4620654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4622472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4624330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4626131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4627919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4629740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4631590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4633391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4635196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4636966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4638764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4640555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4642341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4644111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4645949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4647792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4649563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4651353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4653141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4655039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4656859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4658880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4660744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4662708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4664427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4666277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4668114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4670100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4671993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4673846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4675678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4677544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4679424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4681333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4683153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4684992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4686822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4688638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4690485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4692333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4694218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4696090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4697933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4699768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4701602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4703461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4705352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4707187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4709021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4710847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4712700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4714549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4716372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4718258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4720137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4721998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4723812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4725655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4727510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4729410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4731221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4733056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4734889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4736747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4738563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4740401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4742266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4744174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4745989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4747880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4749651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4751520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4753437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4755261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4757105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4759170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4761021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4762852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4764690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4766601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4768504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4770326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4772162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4774042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4775939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4777741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4779574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4781426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4783227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4785059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4786889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4788732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4790572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4792450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4794286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4796144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4797956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4799833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4801713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4803573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4805396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4807252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4809086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4810956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4812775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4814637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4816521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4818352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4820172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4822026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4823914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4825754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4827584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4829402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4831250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4833086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4834918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4836736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4838615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4840501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4842335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4844154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4846031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4847936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4849759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4851601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4853484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4855343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4857168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4859274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4861118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4863038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4864912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4866745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4868568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4870512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4872386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4874222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4876044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4877902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4879718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4881544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4883390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4885219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4887087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4888955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4890800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4892631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4894508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4896385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4898222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4900061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4901892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4903710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4905539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4907368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4909183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4911031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4912895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4914747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4916569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4918456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4920307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4922160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4923964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4925799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4927627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4929481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4931300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4933145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4935023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4936931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4938755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4940602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4942482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4944385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4946251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4948054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4949890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4951721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4953588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4955413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4957260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4959433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4961367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4963190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4965030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4966927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4968831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4970646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4972487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4974324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4976151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4977970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4979797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4981654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4983507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4985391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 41%] 2024-08-08T21:00:10.4987216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.4989065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.4990919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.4992804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.4994627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.4996470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.4998274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5000106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5001924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5003775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5005584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5007460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5009328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5011165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5012989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5014845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5016724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5018557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5020385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5022201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5024045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5025882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5027734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5029546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5031433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5033320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5035154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5036972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5038848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5040780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5042581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5044407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5046225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5048068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5049875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5051715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5053588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5055501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5057356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5059456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5061291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5063236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5065113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5066944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5068776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5070636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5072443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5074262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5076110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5077947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5079837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5081722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5083550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5085369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5087228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5089095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5090928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5092756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5094586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5096396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5098230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5100064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5101892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5103747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5105633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5107460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5109268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5111146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5113016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5114866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5116672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5118504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5120317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5122161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5123970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5125809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5127659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5129540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5131329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5133151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5135026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5136914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5138732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5140534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5142363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5144192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5146002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5147814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5149651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5151521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5153394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5155255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5157104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5159280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5161205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5163021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5164867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5166708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5168541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5170358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5172177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5174020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5175872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5177769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5179586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5181431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5183281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5185162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5186978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5188828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5190630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5192464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5194279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5196133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5197944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5199819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5201697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5203545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5205398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5207251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5209140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5210978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5212808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5214624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5216468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5218291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5220108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5221920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5223798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5225698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5227504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5229322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5231220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5233100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5234902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5236734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5238559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5240412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5242224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5244060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5245890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5247786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5249648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5251486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5253314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5255255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5257117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5259210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5261043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5262886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5264684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5266511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5268335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5270163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5272062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5273945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5275780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5277605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5279484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5281359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5283190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5285019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5286846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5288658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5290495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5292336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5294161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5296020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5297904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5299732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5301534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5303406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5305266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5307097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5308890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5310708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5312521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5314362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5316169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5318000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5319852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5321759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5323555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5325380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5327247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5329145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5330957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5332772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5334615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5336455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5338285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5340107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5341949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5343827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5345710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5347530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5349362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5351234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5353094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5354945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5356785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5358822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5360667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5362485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5364308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5366156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5368029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5369933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5371757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5373605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5375473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5377371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5379195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5381059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5382873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5384711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5386536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5388398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5390212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5392085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5393952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5395797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5397594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5399446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5401323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5403158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5404977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5406795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5408633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5410464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5412289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5414106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5415999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5417870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5419698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5421512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5423445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5425320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5427144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5428960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5430783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5432632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5434435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5436266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5438071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5439942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5441777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5443601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5445418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5447301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5449140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5450965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5452776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5454658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5456459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5458285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5460378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5462217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5464122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5466014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5467854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5469689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5471586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5473470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5475308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5477144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5478973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5480792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5482663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5484458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5486272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5488122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5490003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5491834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5493637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5495515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5497391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5499239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5501047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5502875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5504687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5506525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5508335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5510171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5512041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5513938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5515742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5517575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5519434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5521339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5523144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5524962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5526805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5528600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5530413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5532215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5534051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5535911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5537780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5539599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5541431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5543292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5545149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5546956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5548787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5550610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5552434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5554288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5556127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5557981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5560082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5562011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5563833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5565684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5567549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5569451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5571253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5573086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5574885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5576706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5578525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5580391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5582175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5584151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5585907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5587731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5589545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5591353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5593234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5595102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5596921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5598742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5600581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5602411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5604241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5606059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5607934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5609815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5611662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5613456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5615281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5617132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5618988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5620789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5622613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5624454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5626252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5628079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5629893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5631778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5633617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5635441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5637250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5639089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5640926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5642801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5644619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5646465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5648282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5650088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5651933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5653775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5655636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5657499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5659536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5661368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5663175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5665039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5666930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5668747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5670562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5672371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5674209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5676018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5677850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5679689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5681598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5683422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5685231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5687066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5688935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5690832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5692636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5694482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5696307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5698152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5699966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5701877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5703645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5705526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5707316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5709139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5710957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5712880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5714708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5716522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5718361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5720190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5722004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5723812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5725672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5727563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5729417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5731236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5733088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5734930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5736817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5738689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5740536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5742389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5744225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5746049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5747871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5749718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5751565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5753467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5755319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5757176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5759224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5761159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5763724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5765583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5767403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5769244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5771066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5772928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5774742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5776665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5778563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5780413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5782248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5784072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5785970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5787858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5789697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5791529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5793370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5795198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5797037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5798839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5800731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5802683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5804515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5806331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5808177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5810051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5811934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5813760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5815590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5817441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5819249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5821086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5822912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 42%] 2024-08-08T21:00:10.5824802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5826664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5828507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5830332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5832192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5834048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5835934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5837749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5839600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5841400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5843235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5845062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5846883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5848766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5850614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5852456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5854332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5856157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5858009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5860185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5862033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5863870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5865705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5867557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5869404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5871244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5873167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5875063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5876900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5878728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5880569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5882447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5884360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5886161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5887997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5889823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5891683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5893501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5895335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5897201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5899121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5900911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5902747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5904571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5906467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5908327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5910166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5912010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5913860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5915703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5917532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5919376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5921255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5923137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5924962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5926800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5928630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5930485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5932345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5934189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5936036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5937862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5939680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5941530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5943366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5945216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5947102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5948926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5950773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5952635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5954598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5956399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5958256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5960326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5962189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5964017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5965864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5967668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5969578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5971462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5973303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5975100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5976971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5978872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5980702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5982526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5984343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5986172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5987997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5989834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5991638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5993545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5995408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5997244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.5999063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6000981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6002855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6004687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6006600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6008362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6010211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6012032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6013881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6015702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6017591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6019448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6021277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6023103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6025003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6026865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6028702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6030526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6032379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6034193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6036028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6037849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6039699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6041547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6043434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6045278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6047107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6048984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6050849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6052709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6054570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6056406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6058226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6060329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6062165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6064076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6065873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6067775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6069608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6071431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6073314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6075197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6077043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6078854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6080677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6082503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6084346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6086158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6087997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6089865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6091776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6093592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6095439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6097306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6099217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6101027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6102863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6104673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6106514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6108314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6110122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6111962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6113834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6115702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6117520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6119358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6121226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6123096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6124905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6126738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6128563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6130387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6132203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6134052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6135895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6137776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6139644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6141474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6143325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6145189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6147071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6148892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6150741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6152535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6154409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6156231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6158107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6160110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6162026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6163918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6165769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6167572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6169462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6171349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6173171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6175115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6176808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6178651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6180481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6182309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6184131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6186052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6187936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6189765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6191579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6193449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6195326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6197129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6198941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6200771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6202597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6204396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6206227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6208041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6209934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6211774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6213597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6215418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6217308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6219179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6221012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6222843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6224697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6226511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6228351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6230178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6232036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6233889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6235759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6237589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6239410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6241270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6243125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6244954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6246782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6248608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6250422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6252256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6254133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6255955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6257809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6259913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6261755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6263578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6265508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6267402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6270988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6274594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6278150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6281697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6285171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6288739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6292295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6295892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6299472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6302978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6306468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6310037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6313643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6317147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6320638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6324217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6337165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6340997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6344762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6348540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6352443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6356392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6360472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6364274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6368194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6372066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6375883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6379645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6383450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6387225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6390999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6394835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6398623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6402472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6406266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6409925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6413465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6417056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6420630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6424165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6427713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6431238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6434732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6438260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6441804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6445328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6448888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6452468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6456066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6459873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6463496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6467092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6470648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6474170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6477680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6481184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6484679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6488219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6491671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6495257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6498861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6502373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6505856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6509437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6513019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6516543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6520020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6523546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6527089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6530592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6534106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6537648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6541243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6544838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6548357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6551883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6555536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6559381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6562954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6566482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6570011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6573534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6577032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6580539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6584082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6587697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6591279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6594811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6598348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6601917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6605491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6609020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6612557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6616086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6619589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6623113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6626667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6630188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6633759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6637343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6640924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6644457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6648011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6651569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6655135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6658891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6662468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6665996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6669544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6673072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6676557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6680168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6683802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6687321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6690852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6694426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6698014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6701522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6705046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6708552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6712097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6715627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6719139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6722653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6726235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6729816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6733329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6736850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6740380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6743907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6747426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6750962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6754517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6758047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6761789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6765324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6768857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6772449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6775999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6779513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6783062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6786596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6790173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6793751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6797308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6800860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6804386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6807946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6811476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6815015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6818559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6822143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6825672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6829185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6832683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6836241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6839819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6843327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6846846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6850388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6853954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6857492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6861259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6864877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6868455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6871972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6875470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6879002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6882606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6886193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6889696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6893228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6896766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6900289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6903812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6907324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6910881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6914416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6917904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6921415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6924943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6928511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6932056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6935569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6939085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6942592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6946082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6949593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6953151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6956797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6960611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6964149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6967690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6971219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6974805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6978402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6981955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6985477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6988990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6992511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6996037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.6999538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7003093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7006677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7010216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7013707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7017214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7020780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7024417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7027894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7031399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7034898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7038435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7041953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 43%] 2024-08-08T21:00:10.7045457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7049034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7052645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7056213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7059975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7063521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7067139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7070733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7074261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7077791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7081293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7084791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7088287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7091804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7095408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7099000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7102500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7105997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7109546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7113092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7116642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7120146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7123681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7127166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7130669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7134292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7137720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7141291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7144844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7148364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7151878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7155436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7159249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7162833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7166346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7169870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7173331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7176911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7180361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7183843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7187399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7190979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7194520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7197992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7201478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7205082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7208638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7212155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7215661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7219184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7222713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7226255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7229768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7233344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7236930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7240457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7243968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7247471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7251037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7254631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7258147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7261915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7265476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7268977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7272471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7275985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7279581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7283155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7286651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7290158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7293667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7297255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7300818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7304329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7307861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7311365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7314872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7318365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7321902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7325424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7328963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7332516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7336021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7339510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7343015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7346565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7350090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7353593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7357149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7360910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7364436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7367917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7371405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7374983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7378596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7382094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7385581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7389138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7392741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7396286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7399764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7403274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7406810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7410327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7413807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7417315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7420866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7424409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7427890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7431472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7435089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7438647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7442128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7445641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7449149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7452641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7456178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7459863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7463395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7466978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7470542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7474053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7477578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7481154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7484686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7488191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7491711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7495218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7498708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7502205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7505687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7509166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7512681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7516239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7519718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7523217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7526741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7530262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7533778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7537266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7540740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7544237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7547769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7551279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7554801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7558697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7562303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7565863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7569325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7572847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7576438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7580040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7583641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7587070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7590534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7594029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7597519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7601116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7604587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7608160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7611653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7615216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7618676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7622216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7625742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7629287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7632762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7636260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7639753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7643291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7646797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7650363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7654303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7657822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7661622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7665155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7668737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7672367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7675879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7679379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7682849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7686356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7689894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7693395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7696934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7700519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7704028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7707525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7711015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7714598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7718155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7721634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7725129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7728639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7732169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7735686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7739178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7742729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7746271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7749776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7753263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7756808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7760620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7764173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7767620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7771111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7774620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7778132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7781594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7785095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7788647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7792209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7795681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7799177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7803894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7807912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7811514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7815020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7818561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7822108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7825586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7829098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7832624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7836132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7839669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7843222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7846722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7850237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7853737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7857345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7861391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7864891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7868390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7871885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7875402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7878896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7882379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7885986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7889599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7893102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7896592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7900179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7903776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7907283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7910755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7914255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7917768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7921268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7924755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7928260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7931804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7935324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7938791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7942269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7945822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7949366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7952844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7956367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7960272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7963796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7967269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7970764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7974277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7977857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7981408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7984923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7988439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7992017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7995574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.7999081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8002584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8006089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8009582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8013078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8016594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8020071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8023589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8027116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8030628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8034120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8037597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8041150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8044719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8048205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8051674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8055216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8059133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8060997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8062799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8067726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8072331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8074139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8075968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8077832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8079682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8081490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8083308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8085116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8086919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8088719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8090550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8092331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8094216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8096170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8098055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8099856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8101787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8103575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8105299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8107218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8108986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8110717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8112522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8114325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8116156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8117987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8120012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8121863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8124017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8125835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8127759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8130746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8134320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8137771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8141476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8145175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8148899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8152511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8156128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8160252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8163973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8167415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8170846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8174216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8177722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8181030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8184517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8188152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8191869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8195421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8199174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8202813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8206480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8209889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8212806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8214564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8216664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8218472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8220287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8222093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8224141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8225941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8227748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8229577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8231391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8233327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8235225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8237061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8238897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8240731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8242554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8244429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8246258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8248059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8249868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8251805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8253776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8255884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8257821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8260124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8261999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8263800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8265634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8267452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8269295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8271105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8272945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8274761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8276684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8278410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8280237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8282217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8284239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8286046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8287882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8289713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8291548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8293375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8295199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8297045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8298879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8300705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8302527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8304353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8306275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8308163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8309963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8311802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8313631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8315446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8317282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8319106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8320934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8322752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8324567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8326382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8328221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8330107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8332024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8333840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8335693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8337501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8339328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8341136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8342985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8344810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8346614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8348444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8350264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8352050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8353950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8355903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8357723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8359914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8361762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8363589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8365414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8367232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8369040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8370880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8372727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8374548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8376364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8378353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8380325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8382157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8383983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8385838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8387681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8389500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8391344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8393162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8395000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8396794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8398628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8400444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8402381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8404274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8406131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8407931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8409788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8411587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8413426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 44%] 2024-08-08T21:00:10.8415255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8417107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8418932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8420732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8422583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8424419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8426337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8428250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8430086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8431922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8433752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8435571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8437399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8439232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8441050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8442868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8444686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8446519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8448343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8450241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8452148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8453991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8455839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8457684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8459900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8461776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8463590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8465419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8467230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8469073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8470880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8472703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8474670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8476648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8478476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8480279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8482096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8483931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8485736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8487533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8489378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8491192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8493004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8494813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8496636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8498532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8500433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8502244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8504085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8505920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8507744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8509560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8511409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8513250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8515076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8516898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8518724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8520627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8522487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8524423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8526235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8528074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8529874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8531701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8533513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8535364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8537169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8538995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8540820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8542666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8544464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8546367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8548273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8550095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8551921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8553737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8555620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8557452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8559636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8561489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8563326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8565163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8566994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8568805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8570767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8572701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8574512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8576314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8578157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8579985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8581780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8583646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8585432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8587296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8589069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8590909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8592729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8594655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8596544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8598369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8600185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8602044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8603855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8605685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8607504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8609378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8611566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8613707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8616005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8618619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8621627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8623820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8625655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8627426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8629311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8631062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8632933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8634719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8636567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8638341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8640176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8642004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8643820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8645730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8647659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8649498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8651303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8653172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8655025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8656909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8659150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8661008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8662865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8664652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8666491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8668265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8670264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8672245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8674039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8675908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8677685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8679563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8681321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8683129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8684967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8686894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8688595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8690424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8692240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8694110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8696008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8697870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8699695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8701521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8703342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8705144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8706967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8708783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8710564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8712376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8714185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8716017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8717851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8719761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8721618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8723451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8725243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8727066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8728873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8730715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8732524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8734340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8736165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8738018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8739820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8741679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8743631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8745510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8747336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8749150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8750983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8752797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8754661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8756472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8758300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8760613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8762488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8764257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8766189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8768159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8770048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8771847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8773687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8775515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8777310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8779145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8780974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8782824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8784632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8786475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8788289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8790182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8792088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8793973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8795773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8797604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8799408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8801221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8803031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8804876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8806669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8808477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8810314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8812139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8814006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8815902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8817781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8819614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8821427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8823245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8825088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8826924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8828823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8830662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8832400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8834233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8836058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8837929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8839870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8841748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8843539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8845363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8847178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8849018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8850821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8852658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8854465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8856373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8858181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8860489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8862300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8864322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8866247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8868073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8869890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8871718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8873525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8875334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8877169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8878986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8880800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8882616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8884425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8886230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8888122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8890001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8891815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8893628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8895439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8897236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8899061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8900884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8902671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8904488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8906302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8908143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8909939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8911869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8913844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8915730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8917430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8919264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8921074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8922942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8924794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8926569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8928373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8930210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8931996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8933792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8935734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8937655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8939475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8941284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8943116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8944935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8946742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8948548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8950373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8952196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8954016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8955879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8957718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8960206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8962204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8964017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8965842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8967697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8969519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8971332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8973139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8974977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8976763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8978585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8980432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8982261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8984127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8986047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8987855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8989695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8991496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8993318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8995121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8996952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.8998769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9000591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9002404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9004237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9006056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9007949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9009898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9011716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9013544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9015362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9017182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9019006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9020814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9022612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9024438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9026257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9028054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9029874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9031769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9033698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9035490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9037315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9039139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9040988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9042790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9044629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9046454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9048300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9050109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9051954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9053792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9055847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9057751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9060078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9061946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9063772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9065590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9067390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9069223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9071050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9072858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9074665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9076493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9078307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9080315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9082266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9084086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9085905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9087722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9089524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9091338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9093182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9094984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9096827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9098624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9100492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9102296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9104179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9107285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9109119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9111009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9112737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9114544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9116369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9118161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9119983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9121796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9123601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9125419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9127232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9129095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9131003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9132863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9134667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9136525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9138329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9140138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9141953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9143789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9145608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9147422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9149237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9151054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9152906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9154833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9156708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9158971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9160898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9162707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9164523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9166332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9168174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9169958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9171779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9173596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9175455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9177357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9179326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9181217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9183058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9184859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9186676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9188505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9190330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9192151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9193957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9195776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9197583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9199408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9201237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9203175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9205035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9206843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9208647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9210477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9212394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9214095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9215905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9217702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9219535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9221328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9223144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9224998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9226928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9228765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9230582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9232392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9234230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9236023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9237845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9239648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9241470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9243253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9245063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9246862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9248666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9250556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9252446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9254255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9256122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 45%] 2024-08-08T21:00:10.9258002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9260120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9261984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9263845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9265629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9267424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9269253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9271076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9272886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9274839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9276791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9278618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9280415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9282244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9284032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9285865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9287635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9289449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9291255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9293087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9294888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9296696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9298628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9300552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9302340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9304166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9305978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9307811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9309602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9311407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9313237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9315056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9316873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9318761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9320575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9322421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9324331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9326165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9327962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9329778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9331666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9333367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9335254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9337045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9338821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9340626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9342442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9344271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9346141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9348046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9349852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9351687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9353488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9355373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9357194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9359514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9361359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9363187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9365085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9366922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9368639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9370641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9372614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9374491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9376203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9378009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9379834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9381648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9383458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9385263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9387081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9388893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9390705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9392507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9394409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9396309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9398112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9399907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9401737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9403553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9405342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9407184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9408981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9410835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9412617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9414442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9416241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9418109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9419978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9421911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9423705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9425534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9427326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9429134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9430935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9432753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9434556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9436349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9438184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9439995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9441851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9443743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9445624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9447445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9449265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9451076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9452902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9454727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9456601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9458863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9460706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9462537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9464297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9466236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9468204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9470111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9471897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9473712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9475554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9477337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9479124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9480941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9482843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9484590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9486435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9488304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9490008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9491944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9493808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9495613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9497441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9499264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9501071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9502877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9504719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9506501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9508300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9510100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9511922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9513725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9515602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9517481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9519297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9521110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9522947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9524720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9526525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9528354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9530141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9531955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9533762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9535598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9537389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9539390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9541179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9543004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9544798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9546622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9548407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9550222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9552001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9553806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9555687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9557562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9559742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9561567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9563603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9565558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9567357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9569175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9571004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9572821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9574632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9576440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9578274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9580089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9581909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9583723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9585551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9587453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9589330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9591152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9592946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9594772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9596561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9598381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9600188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9602022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9603833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9605626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9607462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9609268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9611145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9613055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9614852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9616685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9618484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9620285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9622112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9623949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9625751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9627557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9629382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9631203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9633018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9634927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9636812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9638639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9640434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9642203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9644023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9645834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9647645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9649453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9651245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9653077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9654924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9656741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9659078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9661148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9663011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9664818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9666636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9668459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9670244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9672072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9673884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9675719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9677537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9679341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9681131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9682975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9684914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9686773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9688590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9690408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9692221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9694015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9695835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9697645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9699440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9701231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9703055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9704866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9706720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9708629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9710460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9712294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9714085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9715911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9717710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9719545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9721342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9723152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9724948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9726774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9728553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9730359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9732233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9734175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9735954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9737771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9739572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9741375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9743182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9744968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9746782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9748584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9750392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9752179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9753989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9755939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9757843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9760115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9761966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9763785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9765581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9767377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9769214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9770990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9772867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9774573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9776373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9778244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9780159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9782098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9783888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9785712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9787502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9789308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9791117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9792951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9794738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9796549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9798366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9800213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9802032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9803918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9805914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9807724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9809535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9811344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9813164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9814961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9816760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9818551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9820365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9822172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9823974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9825773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9827673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9829588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9831372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9833179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_1024_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9835015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9836869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9838678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9840525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9842357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9844212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9846028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9847870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9849687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9851623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9853521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9855403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9857214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9859523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9861358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9863182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9865007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9866839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9868660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9870470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9872307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9874129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9876074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9878034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9879953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9881767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9883597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9885417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9887257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9889093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9890932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9892754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9894648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9896523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9898379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9900625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9902576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9904389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9906197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9908006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9909843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9911666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9913505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9915303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9917149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9918946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9920760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9922561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9924430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9926363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9928231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9930040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9931859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9933705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9935497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9937330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9939149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9940990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9942791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9944626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9946426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9948298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9950180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9952043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9966872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9968918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9970776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9972603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9974450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9976291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9978121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9979954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9981796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9983657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9985643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9987921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9989895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9991767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9993608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9995572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9997298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:10.9999267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0001002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0002947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0004758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0006627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0008422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0010302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0012221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0014127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0015945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0017782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0019627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0021415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0023209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0025038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0026887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0028729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0030580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0032390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0034282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0036216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0038099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0039928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0041785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0043637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0045472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0047297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0049128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0050974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0052841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0054627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0056578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0058885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0060904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0062818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0064648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0066513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0068323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0070160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0071983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0073841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0075655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0077504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0079338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0081176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0083058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0084993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0086900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0088739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0090573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0092399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0094228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0096057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0097881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0099693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0101535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0103372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0105200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 46%] 2024-08-08T21:00:11.0107069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0109012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0110950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0112741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0114535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0116374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0118224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0120043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0121899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0123733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0125593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0127423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0129265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0131142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0133096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0134967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0136806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0138631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0140491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0142297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0144132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0145966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0147800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0149633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0151466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0153311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0155262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0157207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0159599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0161469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0163301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0165112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0166942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0168787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0170634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0172467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0174297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0176150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0177984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0179949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0181933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0183819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0185661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0187471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0189304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0191128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0192979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0194793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0196629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0198454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0200334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0202116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0204003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0205911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0207807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0209667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0211441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0213293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0215134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0216963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0218817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0220635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0222476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0224319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0226134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0228003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0229919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0231782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0233586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0235428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0237270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0239089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0240908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0242736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0244580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0246384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0248222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0250047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0251968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0253874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0255815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0257644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0259988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0261883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0263706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0265531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0267393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0269203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0271045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0272863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0274695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0276622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0278639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0280480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0282318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0284147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0285976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0287815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0289652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0291479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0293291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0295118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0296948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0298772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0300588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0302583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0304511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0306312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0308144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0309968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0311817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0313632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0315480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0317288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0319128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0320931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0322755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0324568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0326479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0328377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0330208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0332033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0333967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0335688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0337497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0339351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0341192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0343005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0344851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0346687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0348525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0351240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0353308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0355046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0356947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0359216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0361140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0363182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0365173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0366980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0368807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0370636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0372490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0374295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0376285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0378329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0380074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0381903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0383727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0385565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0387400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0389237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0391063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0392912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0394761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0396598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0398432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0400349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0402304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0404117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0405949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0407790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0409610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0411410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0413246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0415069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0416922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0418736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0420573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0422392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0424321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0426225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0428056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0429884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0431745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0433550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0435391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0437227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0439119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0440911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0442747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0444603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0446441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0448355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0450269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0452098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0453924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0455811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0457620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0459870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0461744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0463568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0465388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0467495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0469289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0471188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0473109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0475064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0476912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0478719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0480620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0482390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0484340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0486054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0487893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0489813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0491568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0493387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0495223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0497118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0499046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0500840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0502668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0504509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0506324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0508153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0509973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0511832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0513676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0515463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0517341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0519121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0521066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0523006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0524810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0526677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0528553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0530336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0532163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0534020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0535867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0537679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0539530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0541355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0543233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0545067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0547039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0548859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0550692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0552467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0554305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0556185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0558050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0560316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0562149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0564034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0565833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0567640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0569632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0571665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0573463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0575298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0577129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0578976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0580812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0582690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0584477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0586304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0588131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0589956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0591755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0593665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0595582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0597408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0599228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0601075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0602906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0604704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0606535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0608354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0610198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0612004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0613846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0615687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0617610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0619503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0621335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0623161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0625013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0626852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0628664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0630475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0632318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0634111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0635919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0637760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0639586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0641443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0643345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0645240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0647070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0648899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0650711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0652554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0654387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0656266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0658088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0660311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0662192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0664025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0665944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0667911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0669841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0671657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0673501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0675330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0677173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0678977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0680838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0682638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0684484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0686302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0688170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0690053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0692000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0693865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0695703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0697523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0699465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0701288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0703007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0704855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0706695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0708655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0710374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0712209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0714097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0716013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0717881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0719705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0721526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0723350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0725155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0726995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0728822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0730618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0732447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0734271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0736118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0737967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0739884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0741744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0743606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0745392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0747228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0749056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0750945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0752740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0754575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0756450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0758320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0760577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0762399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0764391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0766332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0768133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0769955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0771786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0773631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0775423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0777242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0779070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0780901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0782722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0784627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0786356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0788358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0790179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0791986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0793816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0795736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0797480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0799313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0801141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0802991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0804797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0806639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0808448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0810286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0812184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0814068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0815884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0817730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0819531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0821356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0823172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0825004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0826831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0828646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0830476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0832305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0834125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0836006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0837939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0839755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0841583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0843397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0845234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0847067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0848891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0850711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0852522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0854344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0856193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0858013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0860472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0862491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0864284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0866110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0867930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0869775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0871575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0873397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0875222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0877062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0878868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0880715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0882547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0884497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0886421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0888243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0890084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0891929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0893759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0895581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0897410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0899239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0901050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0902860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0904708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0906532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0908456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0910360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0912172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0914001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0915816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0917649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0919475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0921337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0923172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0925016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0926860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0928737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0930565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0932495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0934415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0936284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0938111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0939962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0941789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0943646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0945455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0947300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0949144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 47%] 2024-08-08T21:00:11.0950987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0952836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0954667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0956665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0959045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0960912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0962737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0964591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0966428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0968260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0970095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0971958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0973798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0975645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0977486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0979340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0981415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0983354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0985198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0987022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0988879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0990684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0992515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0994340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0996208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0998020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.0999868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1001702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1003556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1005448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1007380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1009203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1011058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1012874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1014706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1016571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1018399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1020239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1022088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1023961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1025762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1027595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1029503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1031428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1033250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1035070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1036892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1038737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1040563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1042393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1044228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1046148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1047987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1049723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1051559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1053435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1055447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1057403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1059692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1061580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1063447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1065266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1067118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1068958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1070820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1072644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1074500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1076323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1078273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1080240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1082130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1083975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1085830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1087670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1089492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1091344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1093199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1095027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1096857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1098714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1100549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1102427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1104341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1106245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1108095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1109948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1111784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1113617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1115485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1117316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1119156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1120989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1122867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1124677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1126586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1128518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1130423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1132244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1134097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1135934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1137795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1139614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1141468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1143287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1145124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1146968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1148796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1150691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1152643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1154517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1156402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1158262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1160691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1162532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1164358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1166199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1168019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1169835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1171652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1173493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1175471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1177419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1179322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1181190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1182997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1184801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1186628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1188460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1190323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1192134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1194050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1195816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1197676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1199545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1201482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1203362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1205232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1207059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1208892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1210735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1212572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1214404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1216223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1218069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1219910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1221743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1223612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1225548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1227430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1229263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1231083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1232922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1234749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1236579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1238410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1240240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1242117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1243930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1245769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1247640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1249665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1251543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1253382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1255193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1257091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1259459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1261351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1263191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1265045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1266855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1268706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1270530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1272499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1274462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1276342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1278186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1280026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1281854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1283674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1287276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1290857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1294398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1297984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1301509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1305118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1308639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1312265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1315841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1319373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1322890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1326425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1329935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1333572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1337019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1340532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1344071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1347632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1351150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1354699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1358790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1362501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1366043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1369558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1373105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1376661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1380196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1383732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1387266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1390827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1394376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1397906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1401510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1405159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1408723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1412230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1415767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1419322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1422849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1426349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1429893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1433431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1436960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1440487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1444017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1447615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1451216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1454775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1458811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1462468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1466052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1469528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1473045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1476614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1480138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1483760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1487181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1490706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1494215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1497859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1501502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1505030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1508860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1512389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1515918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1519462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1522977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1526478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1529998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1533554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1537082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1540591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1544213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1547890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1551425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1554967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1560605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1564262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1567822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1571346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1574872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1578404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1581920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1585422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1588944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1592678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1596386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1599898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1603429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1607084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1610501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1614005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1617541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1621082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1624619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1628155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1631691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1635256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1638879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1642477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1646017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1649568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1653120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1656703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1660705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1664245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1667746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1671251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1674784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1678335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1681858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1685544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1689196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1692728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1696255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1699772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1703293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1706958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1710626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1714130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1717674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1721239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1724780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1728305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1731953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1735627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1739147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1742683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1746221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1749739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1753249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1756813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1760863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1764404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1767941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1771469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1774996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1778705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1782348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1785863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1789370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1792918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1796441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1799946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1803493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1807046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1810559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1814078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1817610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1821158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1824778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1829158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1832679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1836200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1839721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1843208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1846732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1850282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1853792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1857358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1861424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1864892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1868414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1872089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1875748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1879296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1882812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1886340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1889886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1893442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1897007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1900543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1904082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1907629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1911192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1914720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1918371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1921997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1925510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1929014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1932537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1936080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1939612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1943170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1946715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1950263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1953789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1957427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1961699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1965471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1969148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1972665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1976207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1979759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1983309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1986914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1990424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1994143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.1997594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2001189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2004705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2008300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2011927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2015529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2019042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2022589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2026123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2029617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2033150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2036686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2040211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2043724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2047250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2050779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2054297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2057945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2062112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2065724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2069262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2072731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2076251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2079794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2083333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2086861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2090381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2093904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2097420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2100897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2104559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2108251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2111853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2115369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2118898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2122416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2125937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2129451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2132990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2136543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2140081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2143612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2147140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2150758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2154396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2158052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2162064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2165644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2169193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2172708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2176240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2179776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2183287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2186787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2190371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2193987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2197543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2201203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2204798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2208333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2211863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2215383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2218879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2222465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2225944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2229445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2232979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2236563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2240072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2243623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2247252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2250869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2254407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2257995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2261981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2265516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 48%] 2024-08-08T21:00:11.2269023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2272529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2276043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2279581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2283109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2286619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2290243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2293928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2297527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2301021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2304548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2308097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2311613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2315286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2319195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2322750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2326292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2330266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2333809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2337425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2341120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2344684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2348192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2351812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2355239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2359238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2362770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2366325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2369837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2373341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2376868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2380410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2383929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2387582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2391229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2394766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2398291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2401878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2405345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2408932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2412424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2415942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2419471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2423030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2426565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2430086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2433692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2437336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2440849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2444347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2447879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2451415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2454927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2458901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2462446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2465977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2469511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2473024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2476536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2480226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2483849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2487350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2490868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2494429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2497977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2501448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2504996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2508486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2512000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2515505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2519081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2522534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2526130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2529682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2533176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2536699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2540211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2543702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2547211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2550744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2554238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2557779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2561781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2565336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2568856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2572536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2576188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2579734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2583267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2586802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2590339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2593888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2597420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2600998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2604460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2608066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2611508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2614993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2618637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2622268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2625783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2629282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2632807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2636351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2639871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2643378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2646885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2650412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2653918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2657490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2661543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2665196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2668854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2672411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2675934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2679467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2682989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2686543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2690004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2693522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2697108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2700487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2704000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2707513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2711067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2714687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2718202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2721743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2725236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2728732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2732237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2735765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2739269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2742746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2746273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2749815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2753332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2756946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2761039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2764654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2768166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2771756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2775193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2778703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2782197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2785698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2789177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2792699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2796213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2799721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2803236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2806895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2810516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2814073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2817592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2821127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2824616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2828103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2831645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2835151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2838643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2842146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2845662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2849185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2852786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2856504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2860353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2863946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2867491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2870962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2874450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2877978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2881477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2884937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2888443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2891961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2895448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2899075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2902701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2906230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2909727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2913233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2916748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2920290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2923826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2927353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2930826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2934362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2938053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2941480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2945076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2948685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2952173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2955654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2959641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2963223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2966721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2970221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2973735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2977241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2980751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2984248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2987754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2991414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2995070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.2998544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3002047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3005597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3009116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3012618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3016136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3019678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3023183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3026683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3030203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3033712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3037255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3040836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3044388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3047887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3051405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3054912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3058887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3062369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3065883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3069368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3072855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3074700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3076496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3078459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3080403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3082311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3084100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3085927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3087741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3089574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3091388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3093192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3094985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3096805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3098589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3100381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3102249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3104163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3106016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3107807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3109626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3111442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3113237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3115042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3116906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3118701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3120512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3122324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3124159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3125982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3127880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3129770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3131578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3133417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3135216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3137035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3138835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3140665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3142448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3144266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3146080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3147916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3149701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3151600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3153496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3155335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3157180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3159373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3161205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3163019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3164843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3166632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3168452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3170271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3172091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3173893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3175850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3177788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3179593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3181395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3183219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3185028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3186831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3188632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3190440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3192267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3194085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3195870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3197667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3199593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3201460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3203277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3205082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3206919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3208710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3210522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3212336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3214189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3215984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3217807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3219623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3221442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3223294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3225194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3227056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3228865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3230667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3232463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3234306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3236097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3237908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3239695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3241518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3243341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3245136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3246972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3248879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3250737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3252520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3254332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3256151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3258032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3260198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3262050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3263853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3265684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3267485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3269312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3271103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3273051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3274944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3276754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3278559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3280383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3282184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3283976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3285799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3287613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3289412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3291215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3293040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3294847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3296831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3298624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3300442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3302257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3304069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3305865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3307685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3309607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3311288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3313093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3314885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3316709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3318491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3320367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3322246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3324069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3325853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3327665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3329462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3331281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3333062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3334884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3336682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3338521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3340317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3342109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3344017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3345924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3347731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3349532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3351366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3353185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3354998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3356847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3359024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3360860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3362661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3364460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3366278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3368180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3370107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3371998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3373795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3375627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3377424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3379238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3381070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3382939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3384745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3386574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3400446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3402442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3404514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3406579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3408605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3410581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3412547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3414517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3416486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3418448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3420408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3422340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3424319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3426291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3428229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3430225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3432266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3434272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3436184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3438134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3440095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3442065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3444015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3446010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3447956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3449955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3451887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3453872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3455878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3458015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3460346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3462225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3464062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3465913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3467715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3469533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3471384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3473215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3475041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3476871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3478711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3480652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3482584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3484490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3486348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3488152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3489978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3491895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3493635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3495473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3497294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3499107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3500940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3502795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3504604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 49%] 2024-08-08T21:00:11.3506520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3508413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3510246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3512045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3513876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3515690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3517533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3519337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3521164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3523014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3524831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3526634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3528459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3530433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3532356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3534182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3535999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3537854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3539703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3541528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3543361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3545208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3547045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3548880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3550707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3552540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3554453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3556407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3558227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3560499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3562352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3564160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3565998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3567825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3569676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3571482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3573326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3575150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3576995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3578953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3580911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3582741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3584602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3586423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3588263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3590086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3591955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3593771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3595625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3597434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3599262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3601075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3602957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3604887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3606715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3608537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3610363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3612195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3614028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3615851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3617674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3619508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3621338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3623168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3624985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3626884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3628818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3630624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3632464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3634288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3636134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3637944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3639787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3641600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3643436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3645233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3647058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3648870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3650798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3652689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3654521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3656338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3658222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3660425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3662249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3664091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3665932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3667762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3669585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3671442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3673278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3676660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3678663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3680510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3682358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3684201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3686028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3687866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3689693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3691493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3693324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3695151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3697011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3698812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3700734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3702640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3704483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3706286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3708127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3709943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3711783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3713587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3715422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3717248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3719100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3720910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3722724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3724638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3726555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3728380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3730197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3732030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3733854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3735669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3737461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3739299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3741117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3742933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3744752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3746583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3748491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3750404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3752208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3754018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3755864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3757728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3759967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3761831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3763697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3765498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3767333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3769161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3771007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3772925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3774906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3776783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3778627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3780427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3782275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3784066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3785897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3787724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3789535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3791388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3793208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3795029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3796884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3798810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3800683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3802501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3804320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3806169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3808006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3809836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3811659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3813496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3815357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3817142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3818980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3820837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3822771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3824618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3826442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3828264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3830116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3831912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3833751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3835583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3837426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3839224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3841059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3842874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3844755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3846652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3848517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3850350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3852178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3854010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3855817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3857703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3859952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3861804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3863621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3865450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3867264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3869063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3870994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3872931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3874748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3876559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3878374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3880182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3882022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3883823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3885637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3887453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3889305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3891105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3893229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3895540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3898858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3900731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3902503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3904329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3906178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3907984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3909828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3911640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3913460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3915280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3917083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3918907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3920802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3923800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3925692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3927539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3929373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3931185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3932993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3934836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3936666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3938485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3940338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3942177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3944025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3945956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3947887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3949702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3951553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3953369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3955209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3957066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3959286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3961117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3962939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3964759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3966609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3968405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3970379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3972317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3974140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3976108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3977930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3979774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3981604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3983427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3985244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3987085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3988919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3990743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3992563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3994476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3996391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.3998214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4000032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4001865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4003688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4005475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4007300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4009127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4010967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4012799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4014617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4016432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4018354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4020239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4022061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4023878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4025717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4027510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4029333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4031159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4033018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4034820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4036636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4038476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4040295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4042176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4044085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4045943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4047761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4049573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4051374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4053193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4055007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4056873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4059030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4060899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4062753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4064545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4066429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4068372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4070282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4072079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4073914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4075749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4077597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4079403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4081239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4083056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4084907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4086722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4088556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4090408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4092336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4094177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4095999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4097817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4099652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4101467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4103279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4105119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4106941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4108760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4110571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4112405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4114273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4116170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4118037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4119877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4121718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4123549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4125355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4127188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4129027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4130834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4132668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4134490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4136325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4138117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4140011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4141913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4143753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4145555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4147387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4149199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4151047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4152858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4154678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4156486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4158737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4160581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4162399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4164367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4166347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4168134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4170333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4172189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4174028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4175859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4177651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4179480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4181299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4183101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4184917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4186727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4188765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4190971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4192694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4194537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4196361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4198155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4199987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4201790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4203636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4205432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4207256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4209090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4210937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4212831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4214781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4216590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4218444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4220270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4222099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4223914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4225738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4227549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4229351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4231197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4233016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4234834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4236807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4238910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4241247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4243188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4245034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4246870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4248902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4250765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4252578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4254414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4256438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4258390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4260494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4262485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4264450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4266238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4268065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4269893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4271712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4273500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4275331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4277134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4278971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4280768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4282621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4284425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4286277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4288528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4290538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4292406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4294236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4296368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4298214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4300055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4301908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4303717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4305535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4307376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4309268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4311084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4313010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4314885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4316701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4318490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4320402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4322118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4323957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4325757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4327583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4329390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4331232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4333048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4334884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4336778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4338662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4340451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4342281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4344104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4345944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4347750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4349569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4351398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 50%] 2024-08-08T21:00:11.4353227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4355058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4356914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4359113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4361107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4363033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4364827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4366654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4368480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4370287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4372092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4373930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4375748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4377558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4379364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4381179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4383013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4384882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4386808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4388596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4390435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4392239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4394058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4395859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4397703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4399497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4401316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4403121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4404948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4406724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4408625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4410511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4412326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4414136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4415939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4417760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4419578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4421393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4423194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4425019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4426843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4428662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4430465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4432375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4434283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4436101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4437912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4439730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4441576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4443375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4445209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4447019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4448849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4450636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4452461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4454263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4456172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4458090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4460292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4462132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4463978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4465772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4467590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4469400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4471232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4473046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4474854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4476701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4478524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4480437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4482379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4484295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4486119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4487944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4489761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4491591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4493406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4495221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4497028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4498855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4500688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4502480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4504353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4506259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4508157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4509953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4511770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4513587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4515418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4517216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4519049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4520866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4522708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4524520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4526352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4528215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4530152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4532003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4533823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4535648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4537467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4539267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4541066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4542898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4544718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4546531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4548347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4550174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4551988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4553874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4555765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4557628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4559815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4561663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4563461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4565271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4567120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4568919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4570738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4572554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4574390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4576182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4578131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4580062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4581880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4583768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4585492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4587295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4589124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4590918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4592711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4594528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4596340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4598152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4599949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4601857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4603766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4605581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4607395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4609234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4611063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4612879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4614690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4616521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4618339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4620151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4621964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4623775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4625679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4627546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4629366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4631176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4633019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4634838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4636641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4638450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4640292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4642081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4643900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4645711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4647540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4649381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4651294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4653156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4654987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4656851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4659033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4660877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4662712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4664535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4666346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4668170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4669985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4671788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4673667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4675677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4677561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4679373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4681184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4683018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4684845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4686643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4688475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4690279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4692115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4693922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4695738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4697594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4699527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4701370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4703186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4705000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4706843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4708628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4710461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4712275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4714079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4715879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4717685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4719508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4721323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4723212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4725091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4726910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4728728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4730542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4732339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4734162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4735989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4737813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4739627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4741470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4743302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4745106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4747010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4748912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4750750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4752554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4754393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4756198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4758071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4760246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4762093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4763903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4765740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4767542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4769365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4771321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4773283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4775073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4776880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4778715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4780519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4782328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4784134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4785963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4787780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4789593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4791401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4793224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4795117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4797016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4798816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4800635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4802450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4804254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4806043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4807843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4809684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4811470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4813286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4815096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4816928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4818754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4820655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4822500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4824335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4826130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4827947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4829752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4831597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4833391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4835192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4837015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4838840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4840652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4842503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4844405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4846271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4848062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4849859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4851686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4853502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4855306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4857152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4859319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4861160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4862967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4864769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4866669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4868629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4870490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4872303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4874117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4875966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4877761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4879588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4881403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4883241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4885033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4886868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4888695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4890494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4892463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4895190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4896995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4898820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4900639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4902438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4904261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4906085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4907893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4909689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4911513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4913335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4915136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4917008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4918918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4920734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4922545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4924357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4926157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4927995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4929790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4931611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4933397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4935218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4937007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4938804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4940679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4942605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4944391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4946196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4948003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4949836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4951608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4953422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4955231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4957096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4959272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4961112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4962945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4964879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4966801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4968676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4970509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4972334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4974145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4975946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4977760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4979571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4981371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4983168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4985003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4986819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4988649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4990555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4992416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4994242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4996039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4997860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.4999700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5001571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5003400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5005245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5007079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5008954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5010784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5012679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5014600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5016542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5018348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5020183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5022042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5023881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5025712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5027541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5029398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5031249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5033094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5034922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5036811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5038758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5040644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5042471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5044328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5046186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5048025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5049855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5051703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5053570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5055399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5057306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5059498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5061579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5063545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5065463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5067292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5069154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5070985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5072832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5074669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5076543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5078364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5080214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5082059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5083923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5085802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5087720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5089614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5091455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5093293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5095132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5096984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5098836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5100682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5102509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5104366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5106221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5108068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5109963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5111908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5113791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5115622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5117441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5119282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5121140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5122958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5124815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5126645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5128506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5130320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5132162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5134053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5136021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5137891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5139737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5141579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5143454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5145274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5147132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5148984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5150839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5152687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5154525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5156370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5158306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5160637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5162538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5164392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5166246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5168084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5169907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5171764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5173618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5175461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5177293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5179156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5180999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5182892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5184829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5186724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5188597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 51%] 2024-08-08T21:00:11.5190434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5192290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5194126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5195998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5197835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5199681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5201517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5203378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5205185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5207095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5209028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5210889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5212711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5214564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5216394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5218261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5220078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5221913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5223758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5225592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5227432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5229256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5231192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5233133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5234969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5236795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5238654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5240499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5242337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5244171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5246016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5247848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5249656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5251504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5253329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5255261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5257207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5259447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5261313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5263175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5264993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5266834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5268673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5270543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5272359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5274204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5276052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5277920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5279872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5281848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5283687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5285538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5287395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5289234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5291078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5292922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5294758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5296574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5298427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5300282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5302121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5304031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5305988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5307833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5309669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5311506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5313356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5315201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5317022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5318877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5320705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5322571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5324409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5326245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5328163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5330116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5331935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5333785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5335621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5337474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5339272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5341112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5342945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5344799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5346619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5348452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5350298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5352209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5354132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5355957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5357842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5360426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5362269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5364093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5365948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5367806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5369643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5371468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5373323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5375167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5377153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5379105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5380947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5382784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5384593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5386428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5388251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5390109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5391931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5393773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5395602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5397465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5399270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5401200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5403108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5404961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5406779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5408627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5410466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5412328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5414154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5416003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5417836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5419680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5421523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5423346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5425263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5427247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5429067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5430883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5432736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5434578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5436410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5438239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5440085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5441915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5443748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5445572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5447394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5449315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5451206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5453042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5454865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5456738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5459096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5460870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5462701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5464566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5466380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5468229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5470052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5471895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5473820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5475765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5477580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5479412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5481272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5483078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5484914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5486746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5488572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5490475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5492234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5494084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5495919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5497835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5499775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5501619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5503459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5505293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5507147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5508994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5510815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5512667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5514486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5516338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5518153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5519982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5521886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5523822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5525634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5527478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5529312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5531173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5532983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5534836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5536668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5538526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5540346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5542205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5544043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5545973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5547903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5549732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5551582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5553437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5555280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5557143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5559358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5561228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5563052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5564875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5566730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5568556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5570523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5572483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5574325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5576165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5577983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5579826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5581649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5583608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5585338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5587175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5589012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5590882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5592696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5594613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5596534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5598395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5600212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5602065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5603888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5605733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5607536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5609362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5611207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5613047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5614890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5616712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5618630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5620557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5622384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5624211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5626055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5627890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5629714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5631534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5633385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5635223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5637067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5638896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5640719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5642649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5644549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5646404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5648199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5650051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5651873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5653672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5655499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5657398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5659542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5661453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5663284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5665129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5667057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5669033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5670860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5672705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5674554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5676383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5678229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5680084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5681930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5683759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5685612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5687465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5689305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5691207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5693159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5694958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5696779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5698605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5700454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5702290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5704124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5705962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5707787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5709653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5711481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5713319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5715224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5717161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5718974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5720812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5722653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5724519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5726341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5728191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5730032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5731894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5733719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5735573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5737393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5739308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5741216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5743028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5744871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5746719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5748554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5750376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5752232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5754080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5755911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5757780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5759979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5761836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5763807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5765754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5767599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5769440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5771261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5773103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5774929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5776791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5778616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5780456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5782274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5784126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5785922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5787820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5789731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5791577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5793384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5795230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5797055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5799014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5800715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5802544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5804391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5806231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5808072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5809893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5811785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5813790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5815681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5817511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5819361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5821206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5823151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5824871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5826716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5828548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5830371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5832190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5834062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5835908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5837809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5839708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5841529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5843393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5845301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5847045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5848864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5850722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5852531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5854370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5856202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5858102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5860349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5862358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5864254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5866120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5867955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5869790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5871625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5873456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5875282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5877091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5878937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5880783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5882610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5884469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5886395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5888276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5890106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5891936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5893782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5895613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5897447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5899273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5901099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5902963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5904785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5906631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5908504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5910454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5912322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5914173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5915999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5917845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5919642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5921476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5923311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5925166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5926980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5928826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5930649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5932525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5934439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5936324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5938156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5939996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5941832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5943654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5945507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5947366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5949186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5951013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5952868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5954706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5956587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5958892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5960879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5962709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5964512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5966348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5968169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5970022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5971838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5973672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5975492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5977353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5979155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5981057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5982982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5984875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5986673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5988510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5990339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5992193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5993999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5995826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5997658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.5999504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6001324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6003150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6005033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6006928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6008799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6010598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6012432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6014269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6016085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6017897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6019739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6021564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6023384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6025192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6027024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6028874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6030755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6032691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 52%] 2024-08-08T21:00:11.6034522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6036382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6038200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6040039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6041867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6043730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6045540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6047383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6049201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6051044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6052838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6054736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6056641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6058818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6060688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6062519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6064358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6066190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6068021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6069834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6071694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6073544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6075382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6077210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6079202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6081171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6083016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6084861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6086726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6088586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6090419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6092287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6094238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6095978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6097801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6099645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6101477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6103414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6105321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6107166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6108999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6110870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6112692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6114564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6116379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6118249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6120070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6121922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6123774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6125666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6128476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6130363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6132218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6134080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6135937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6137772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6139614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6141461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6143294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6145116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6146974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6148828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6150712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6152625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6154529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6156375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6158255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6160486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6162317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6164185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6166009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6167849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6169682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6171552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6173380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6175327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6177291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6179218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6181038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6182884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6184716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6186565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6188370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6190189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6192030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6193864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6195700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6197528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6199411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6201403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6203275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6205093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6206948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6208797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6210635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6212465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6214333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6216182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6218029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6219867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6221719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6223629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6225546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6227447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6229274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6231133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6232948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6234787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6236625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6238494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6240311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6242160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6244001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6245859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6247724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6249653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6251534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6253397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6255215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6257044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6259292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6261219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6263085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6264919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6266776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6268639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6270478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6272394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6274365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6276263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6278095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6279925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6281783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6283628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6285476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6287311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6289164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6291016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6292845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6294688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6296567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6300285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6304003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6307444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6311008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6314663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6318257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6321710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6325278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6328850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6332412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6335966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6339571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6343106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6346720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6350277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6353838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6357419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6361354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6364895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6368422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6371995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6375536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6379077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6382625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6386194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6389835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6393482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6397120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6400701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6404267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6407822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6411387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6414968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6418522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6422070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6425623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6429191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6432780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6436374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6440017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6443613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6447168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6450717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6454275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6457896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6461824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6465384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6468913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6472486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6476034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6479618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6483203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6486923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6490588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6494069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6497640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6501212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6504760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6508308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6511855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6515384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6518902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6522433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6525969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6529610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6533257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6536785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6540319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6543884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6547430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6550944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6554494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6558106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6562071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6565596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6569164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6572733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6576425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6580090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6583648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6587196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6590766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6594316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6597850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6601400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6604938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6608452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6611970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6615534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6619079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6622689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6626318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6629872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6633406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6636922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6640485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6644064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6647587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6651127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6654758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6658303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6662191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6665739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6669423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6673119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6676674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6680215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6683757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6687304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6690832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6694354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6697897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6701453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6704995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6708517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6712056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6715703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6719317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6722955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6726392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6729937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6733459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6736978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6740514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6744087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6747622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6751166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6754728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6758309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6762316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6765990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6769589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6773128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6776660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6780150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6783683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6787237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6790780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6794290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6797827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6801389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6804895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6808468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6812102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6815734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6819268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6822810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6826371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6829922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6833476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6837031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6840580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6844150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6847718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6851234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6854902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6858958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6862644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6866165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6869721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6873282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6876812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6880344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6883906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6887466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6891008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6894550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6907558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6911305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6914999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6918600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6922161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6925776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6929314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6932831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6936407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6939985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6943539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6947075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6950626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6954170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6957800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6961938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6965559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6969127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6972683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6976218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6979757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6983303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6986851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6990389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6993919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.6997481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7001018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7004627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7008290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7011908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7015454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7018999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7022573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7026126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7029659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7033213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7036755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7040304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7043844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7047347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7050925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7054571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7058325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7062157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7065711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7069277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7072806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7076336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7079864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7083424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7086990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7090502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7094041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7097692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7101368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7104944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7108503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7112082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7115627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7119160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7122707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7126229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7129745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7133258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7136795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7140336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7143923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7147532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7151093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7154638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7158257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7162191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7165729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7169288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7172846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7176350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7179905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7183493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7187037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7190663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7194351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7197988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7201531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7205070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7208616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7212155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7215690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7219213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7222717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7226280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7229830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7233353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7236939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7240614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7244212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7247717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7251262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7254824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7258766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7262284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7265847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7269408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7272944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7276495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7280045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7283700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7287380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7290973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7294527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7298063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7301596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7305117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7308656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7312225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7315756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7319265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7322819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7326368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7329966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7333591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7337176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7340729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7344252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7347831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7351302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7354873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7358808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7362385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7365930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7369486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7373024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7376647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7380334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7383910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7387423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7390908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7394465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7397999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7401535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7405071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7408601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7412145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7415662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7419172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7422774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 53%] 2024-08-08T21:00:11.7426428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7430031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7433551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7437129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7440668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7444220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7447762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7451322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7454892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7458827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7462462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7465993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7469626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7473291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7476876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7480420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7483978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7487508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7491035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7494585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7498154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7501693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7505230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7508776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7512304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7515888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7519507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7523092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7526659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7530206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7533744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7537296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7540831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7544384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7547909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7551451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7555000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7558895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7562528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7566163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7569778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7573316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7576840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7580378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7583898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7587426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7590946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7594466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7598028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7601572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7605086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7608661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7612311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7615904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7619422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7622973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7626543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7630083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7633597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7637133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7640659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7644181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7647692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7651224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7654799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7658880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7662535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7666054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7669605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7673140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7676646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7680160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7683722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7687254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7690770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7694329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7697900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7701520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7705148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7708742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7712291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7715847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7719388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7722916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7726454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7729970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7733478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7737008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7740563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7744097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7747663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7751285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7754888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7758781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7762369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7765906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7769447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7772972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7776476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7780004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7783550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7787084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7790603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7794120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7797799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7801431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7804944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7808465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7811993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7815501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7818996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7822529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7826043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7829562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7833074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7836597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7840125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7843726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7847307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7850827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7854374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7857968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7861882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7865434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7869005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7872541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7876068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7879615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7883175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7886706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7890377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7894034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7897559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7901083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7904605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7908130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7911679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7915225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7918727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7922267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7925809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7929342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7932853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7936484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7940125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7943625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7947145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7950692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7954243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7957817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7961762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7965310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7968842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7972382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7975915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7979436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7983118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7986757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7990234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7993746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.7997282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8000796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8004303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8007828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8011361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8014853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8018361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8021884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8025423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8029026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8032627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8036154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8039689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8043232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8046748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8050280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8053830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8057376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8061356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8064897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8068431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8071930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8075555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8079192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8082728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8086223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8089731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8093257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8096788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8098601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8100435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8102246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8104087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8105895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8107726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8109650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8111583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8113388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8115218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8117048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8118893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8120694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8122528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8124346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8126176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8127968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8129771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8131592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8133453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8135347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8137202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8139031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8140858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8142670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8144460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8146306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8148143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8149966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8151785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8153639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8155471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8157339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8159722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8161648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8163492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8165305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8167144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8168952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8170793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8172599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8174420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8176233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8178078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8179870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8181755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8183690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8185593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8187390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8189202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8191043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8192867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8194683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8196501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8198342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8200171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8201997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8203814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8205686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8207603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8210193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8212010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8213837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8215662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8217470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8219279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8221103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8222948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8224754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8226584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8228400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8230237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8232113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8234024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8235833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8237670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8239474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8241298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8243120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8244978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8246777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8248609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8250430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8252257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8254069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8256012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8257972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8260146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8261986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8263796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8265625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8267443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8269259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8271066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8272891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8274718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8276530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8278328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8280297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8282262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8284063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8285901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8287767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8289598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8291404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8293254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8295075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8296929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8298741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8300592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8302400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8304363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8306245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8308069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8309834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8311688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8313488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8315301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8317136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8318962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8320773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8322585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8324421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8326249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8328142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8330035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8331876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8333703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8335534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8337348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8339184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8341015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8342835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8344645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8346455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8348296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8350085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8351976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8353868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8355705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8357503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8359722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8361612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8363452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8365256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8367079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8368885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8370725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8372527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8374355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8376262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8378232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8380113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8381924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8383756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8385584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8387395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8389209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8391048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8392886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8394666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8396467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8398283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8400145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8402043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8403890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8405713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8407535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8409328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8411143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8412956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8414797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8416593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8418419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8420243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8422086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8423925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8425840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8427709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8429547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8431347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8433179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8434984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8436800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8438614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8440412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8442265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8444070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8445881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8447683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8449583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8451491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8453306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8455113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8456944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8459127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8461003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8462791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8464616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8466441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8468250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8470068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8471870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8473843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8475757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8477577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8479387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8481222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8482999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8484815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8486631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8488461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8490248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8492067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8493875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8495704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8497567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8499448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8501271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8503090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8504909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8506718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8508555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8510392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8512214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8514029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8515864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8517698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8519513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8521518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8523308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8525119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8526921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8528730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8530545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8532490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8534169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8536001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8537813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8539654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8541460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8543278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8545183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8547151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8548962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8550804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8552646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8554528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8556341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8558236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8560464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8562309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8564153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8565992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8567826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8569796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8571731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8573546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8575386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8577233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8579067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8580891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8582740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8584594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8586412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 54%] 2024-08-08T21:00:11.8588238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8590094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8591927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8593815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8595748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8597585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8599450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8601283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8603132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8604956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8606824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8608664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8610492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8612339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8614173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8615976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8617879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8619789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8621644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8623455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8625284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8627131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8628963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8630794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8632623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8634458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8636295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8638133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8639951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8641924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8643863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8645696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8647519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8649367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8651208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8653038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8654860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8656679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8658893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8660775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8662647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8664444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8666417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8668342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8670181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8672005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8673866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8675673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8677502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8679333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8681201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8683016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8684865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8686712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8688574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8690466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8692502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8694232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8696077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8697928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8699762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8701602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8703438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8705274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8707088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8708931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8710783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8712616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8714516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8716444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8718277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8720110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8721945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8723777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8725628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8727445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8729297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8731158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8733004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8734838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8736685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8738604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8740559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8742376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8744217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8746043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8747899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8749704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8751544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8753376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8755231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8757047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8759317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8761205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8763185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8765126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8766943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8768783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8770618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8772448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8774266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8776111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8777963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8779791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8781619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8783468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8785303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8787192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8789115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8790920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8792757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8794558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8796391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8798211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8800091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8801878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8803711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8805527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8807382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8809178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8811051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8812975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8814886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8816699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8818531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8820383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8822224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8824060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8825901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8827746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8829615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8831434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8833280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8835149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8837068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8838940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8840757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8842598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8844434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8846260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8848086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8849939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8851774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8853580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8855425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8857246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8859572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8861539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8863448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8865284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8867148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8868972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8870812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8872641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8874510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8876326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8878179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8880005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8881844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8883687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8885581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8887469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8889312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8891149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8892974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8894810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8896650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8898483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8900299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8902142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8903987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8905815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8907677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8909619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8911521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8913353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8915187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8917025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8918882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8920708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8922565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8924382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8926227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8928042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8929868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8931731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8933667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8935616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8937405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8939224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8941092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8942895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8944735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8946569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8948400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8950227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8952057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8953901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8955781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8957705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8959978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8961850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8963693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8965533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8967352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8969187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8971017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8972828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8974640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8976484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8978312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8980196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8982156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8984038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8985885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8987693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8989532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8991347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8993194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8995004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8996834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.8998653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9000505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9002307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9004184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9006102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9008053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9009857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9011679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9013508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9015322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9017136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9018948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9020776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9022601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9024426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9026240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9028066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9029967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9031872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9033674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9035519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9037361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9039183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9041004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9042847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9044700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9046509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9048358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9050187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9052038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9053925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9055851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9057658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9060806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9064429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9068073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9071255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9074353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9076225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9078079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9079909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9081744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9083563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9085616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9087611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9089453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9091286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9093122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9094974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9096820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9098668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9100501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9102348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9104197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9106035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9107861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9109789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9111719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9113503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9115342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9117174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9119020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9120826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9122669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9124493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9126348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9128165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9129999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9131817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9133751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9135646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9137487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9139323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9141190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9142996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9144826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9146676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9148518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9150352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9152186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9154020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9155840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9157735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9160069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9161935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9163775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9165605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9167423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9169254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9171083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9172906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9174719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9176540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9178391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9180185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9182160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9184103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9185951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9187757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9189600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9191421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9193267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9195081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9196919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9198728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9200572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9202369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9204192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9206118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9208051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9209861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9211674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9213511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9215333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9217157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9218971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9220817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9222641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9224471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9226297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9228137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9230054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9231970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9233807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9235627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9237469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9239286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9241124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9242950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9244794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9246591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9248419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9250247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9252095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9253975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9255901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9257713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9259982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9261821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9263663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9265483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9267344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9269158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9270996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9272826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9274674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9276498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9278452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9280430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9282269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9284098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9285933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9287769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9289596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9291416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9293233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9295068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9296901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9298736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9300551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9302466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9304377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9306182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9308015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9309839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9311683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9313486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9315324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9317157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9319011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9320819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9322665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9324484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9326426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9328297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9330136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9331949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9333794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9335588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9337390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9339222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9341057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9342870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9344691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9346535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9348355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9350221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9352112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9354002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9355834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9357670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9359873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9361747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9363595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9365436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9367312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9369083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9370950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9372759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9374678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9376633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9378542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9380342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9382180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9384005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9385841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9387644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9389490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9391305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9393153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9394966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9396801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9398658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9400616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9402483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9404290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9406129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9407972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9409791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9411607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9413453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9415283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9417113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9418938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9420767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9422630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9424526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9426377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9428207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9430034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9431839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9433665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 55%] 2024-08-08T21:00:11.9435481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9437328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9439124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9440952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9442774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9444613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9446451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9449069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9450938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9452788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9454595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9456437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9458302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9460521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9462344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9464162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9466083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9467813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9469623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9471509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9473495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9475384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9477202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9479021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9480867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9482786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9484508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9486328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9488163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9490041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9491819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9493636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9495462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9497383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9499274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9501106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9502930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9504786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9506596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9508436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9510252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9512090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9513887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9515729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9517542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9519398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9521256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9523171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9524984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9526820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9528644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9530451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9532280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9534106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9535925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9537735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9539574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9541401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9543217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9545098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9547012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9548835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9550663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9552478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9554299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9556120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9557915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9560137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9561968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9563814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9565611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9567432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9569381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9571339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9573135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9574951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9576782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9578633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9580432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9582276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9584103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9585933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9587760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9589589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9591425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9593341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9595262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9597078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9598900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9600726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9602543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9604349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9606184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9608009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9609820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9611641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9613484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9615312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9617201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9619094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9620913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9622757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9624562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9626399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9628227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9630080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9631896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9633726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9635563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9637403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9639213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9641127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9643029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9644869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9646664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9648473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9650308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9652136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9653960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9655784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9657615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9659825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9661680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9663610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9665404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9667354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9669232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9671043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9672893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9674730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9676552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9678373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9680202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9682047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9683854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9685693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9687495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9689366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9691240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9693107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9694922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9696768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9698574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9700391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9702204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9704049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9705845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9707675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9709499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9711320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9713176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9715069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9716946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9718768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9720585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9722402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9724236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9726065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9727895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9729701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9731524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9733342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9735151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9736985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9738914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9740779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9742565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9744389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9746204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9748037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9749828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9751659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9753474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9755319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9757127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9759368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9761219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9763209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9765136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9766971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9768796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9770658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9772462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9774287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9776120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9777937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9779752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9781645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9783405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9785232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9787172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9789065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9790899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9792731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9794555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9796362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9798194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9800030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9801855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9803670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9805506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9807356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9809155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9811059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9812965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9814821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9816632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9818476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9820279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9822114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9823915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9825742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9827562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9829415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9831231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9833055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9834959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9836870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9838682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9840494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9842343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9844171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9845985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9847804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9849640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9851461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9853283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9855109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9856940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9859283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9861288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9863100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9864922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9866744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9868565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9870359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9872174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9874026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9875803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9877622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9879445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9881278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9883144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9885048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9886862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9888702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9890509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9892343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9894164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9896022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9897834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9899655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9901494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9903336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9905158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9907048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9908951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9910761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9912571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9914383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9916218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9918036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9919854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9921668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9923494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9925312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9927132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9928939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9930816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9932744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9934587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9936403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9938222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9940077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9941872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9943702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9945525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9947368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9949170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9951005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9952802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9954665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9956539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9958581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9960619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9962449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9964270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9966070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9967897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9969733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9971545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9973345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9975181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9977004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9978899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9980834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9982737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9984558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9986376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9988196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9990012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9991860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9993666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9995493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9997297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:11.9999127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0000917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0002723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0004606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0006541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0008331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0010154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0011967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0013799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0015589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0017417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0019229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0021052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0022871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0024679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0026515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0028412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0030305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0032115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0033954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0035782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0037597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0039406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0041235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0043047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0044850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0046653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0048491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0050313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0052180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0054088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0055892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0057731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0059938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0061784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0063593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0065433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0067247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0069039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0070852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0072698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0074485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0076436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0078364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0080199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0081996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0083802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0085623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0087431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0089234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0091034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0092856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0094675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0096487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0098284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0100175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0102089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0103888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0105683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0107522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0109345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0111139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0112962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0114786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0116632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0118432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0120265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0122077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0123960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0125854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0127726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0129526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0131355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0133153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0134961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0136771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0138618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0140409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0142213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0144044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0145860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0147708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0149596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_2048_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0151486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0153321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0155148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0156978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0159241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0161172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0163029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0164853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0166708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0168566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0170406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0172374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0174323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0176235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0178039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0179877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0181712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0183559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0185374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0187230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0189055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0190915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0192765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0194576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0196451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0198411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0200271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0202112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0203949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0205823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0207639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0209467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0211326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0213169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0215010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0216843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0218674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0220547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0222453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0224317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0226165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0228007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0229837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0231653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0233496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0235353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0237174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0239003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0240835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0242684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0244529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0246447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0248322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0250177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0251998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0253844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0255674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0257531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0259787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0261659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0263475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0265328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0267137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0269042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0270988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0272944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 56%] 2024-08-08T21:00:12.0274702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0276528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0278372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0280200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0282024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0283850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0285696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0287537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0289375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0291203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0293113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0295051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0296936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0298764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0300616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0302472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0304291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0306135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0307963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0309815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0311618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0313459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0315286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0317184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0319076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0320966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0322783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0324636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0326451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0328292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0330126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0331991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0333799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0335624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0337476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0339326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0341207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0343121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0345014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0346859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0348700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0350536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0352373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0354207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0356040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0357855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0360137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0362014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0363939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0365760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0367741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0369654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0371463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0373304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0375137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0376984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0378795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0380645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0382466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0384322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0386144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0387989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0389860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0391800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0393659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0395495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0397312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0399158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0400950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0402763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0404616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0406443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0408258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0410076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0411913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0413739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0415634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0417534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0419379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0421217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0423052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0424867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0426717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0428570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0430405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0432229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0434066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0435930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0437795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0439736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0441611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0443434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0445242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0447087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0448907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0450762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0452582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0454416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0456237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0458099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0460334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0462220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0464163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0466132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0467951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0469774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0471622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0473453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0475285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0477118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0478957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0480798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0482717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0484449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0486275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0488169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0490069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0491872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0493723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0495561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0497362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0499194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0501014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0502863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0504668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0506511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0508334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0510178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0512055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0513973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0515798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0517655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0519478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0521321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0523150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0525011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0526821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0528639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0541220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0543197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0545003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0546937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0548846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0550662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0552473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0554284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0556106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0557929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0560185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0562029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0563861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0565688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0567501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0569305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0571268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0573229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0575028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0576859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0578711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0580528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0582332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0584167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0585966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0587792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0589581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0591399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0593205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0595119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0596990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0598806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0600611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0602447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0604264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0606042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0607873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0609689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0611494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0613294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0615118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0616930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0618860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0620767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0622584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0624398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0626212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0628008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0629813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0631616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0633413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0635204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0637010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0638842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0640623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0642527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0644420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0646245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0648037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0649853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0651661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0653495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0655294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0657120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0659346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0661228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0663042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0664870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0666822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0669332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0671152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0672973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0674798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0676602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0678407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0680213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0682045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0683860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0685669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0687474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0689287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0691179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0693150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0694872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0696705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0698535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0700332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0702151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0703972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0705819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0707617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0709452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0711276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0713118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0715004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0716914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0718706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0720526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0722322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0724143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0725952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0727793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0729578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0731377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0733197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0735022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0736856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0738736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0740625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0742440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0744248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0746055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0747890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0749710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0751525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0753343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0755171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0757000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0759200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0761082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0763029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0764986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0766759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0768572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0770389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0772228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0774016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0775837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0777639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0779467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0781259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0783085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0784883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0786783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0788648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0790459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0792266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0794084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0795898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0797703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0799534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0801352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0803182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0804963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0806776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0808575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0810432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0812309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0814170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0815978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0817788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0819585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0821399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0823210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0825001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0826805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0828633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0830461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0832247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0834105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0836003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0837917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0839690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0841524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0843337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0845175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0846971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0848799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0850590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0852410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0854195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0855985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0857859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0860198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0862108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0863907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0865730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0867558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0869368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0871175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0873011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0874830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0876639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0878450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0880280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0882160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0884081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0885955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0887771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0889612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0891415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0893241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0895039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0896874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0898649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0900469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0902298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0904126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0905952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0907853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0909699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0911535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0913327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0915157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0916957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0918770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0920581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0922376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0924204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0926032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0927855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0929692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0931605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0933472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0935284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0937087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0938907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0940706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0942506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0944306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0946128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0947938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0949735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0951550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0953399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0955310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0957137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0959376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0961216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0963069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0964857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0966678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0968492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0970328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0972124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0973954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0975767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0977603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0979537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0981477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0983289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0985095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0986902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0988704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0990533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0992359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0994165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0995964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0997785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.0999600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1001411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1003348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1005249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1007051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1008857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1010656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1012459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1014305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1016096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1017915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1019715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1021563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1023336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1025147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1027009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1028906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1030673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1032483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1034293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1036119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1037904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1039725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1041582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1043335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1045142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1046939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1048756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1050641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1052523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1054309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1056137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1057961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1060178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1062017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1063850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1065668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1067473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1069279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1071093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1072889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1074806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1076741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1078545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1080376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1082172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1083987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1085787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1087624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1089415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1091232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1093041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1094877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1096661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1098552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1100444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1102282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1104081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1105894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1107718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1109538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1111349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1113153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1114962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1116763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1118587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 57%] 2024-08-08T21:00:12.1120357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1122251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1124143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1125945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1127741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1129571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1131383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1133188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1134993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1136798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1138621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1140412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1142236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1144028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1145940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1147811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1149624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1151431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1153270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1155061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1156869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1159043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1160891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1162665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1164470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1166274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1168084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1169979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1171893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1173763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1175565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1177369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1179163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1180981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1182795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1184605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1186400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1188225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1190048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1191860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1193708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1195622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1197480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1199265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1201088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1202885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1204728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1206482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1208300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1210116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1211929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1213719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1215528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1217367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1219275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1221096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1222906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1224726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1226578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1228369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1230183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1232027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1233860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1235683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1237501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1239341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1241208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1243124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1244997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1246813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1248631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1250448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1252246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1254077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1255908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1257718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1261538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1263412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1265258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1267257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1270868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1274516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1277968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1281487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1285080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1288521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1292094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1295648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1299137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1302668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1306217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1309738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1313295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1316900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1320447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1323939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1327440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1330957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1334482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1337996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1341497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1345074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1348525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1352042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1355534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1359549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1363304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1366871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1370347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1373862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1377417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1380966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1384422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1387943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1391447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1394970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1398480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1401976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1405525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1409121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1412650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1416127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1419657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1423167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1426638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1430143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1433671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1437170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1440645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1444176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1447708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1451270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1454868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1458851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1462481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1465999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1469517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1473038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1476580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1480114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1483619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1487117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1490647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1494159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1497746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1501387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1504986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1508496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1511981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1515513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1519040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1522553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1526062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1529574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1533109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1536616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1540126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1543700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1547335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1550904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1554422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1557948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1561906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1565447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1568964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1572487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1575996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1579492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1582973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1586485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1590121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1593778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1597338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1600841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1604369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1607873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1611396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1614892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1618414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1621906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1625397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1628914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1632429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1635989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1639593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1643153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1646674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1650189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1653698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1657181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1661140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1664651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1668196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1671628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1675180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1678647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1682215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1685847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1689435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1692933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1696424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1699956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1703464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1706978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1710497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1714020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1717559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1721094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1724607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1728163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1731801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1735379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1738885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1742404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1745932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1749437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1752916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1756447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1760370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1763913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1767423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1770942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1774560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1778218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1781788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1785296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1788819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1792331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1795824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1799322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1802860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1806372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1809877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1813399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1816930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1820494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1824066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1827617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1831121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1834614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1838104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1841602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1845099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1848604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1852087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1855675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1859481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1863016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1866506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1870121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1873761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1877253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1880737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1884256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1887844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1891312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1894812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1898333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1901859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1905379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1908880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1912377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1915955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1919522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1922998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1926510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1930038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1933531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1937022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1940514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1944033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1947536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1951034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1954540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1958062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1962144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1965724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1969237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1972778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1976305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1979810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1983331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1986878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1990371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1993877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.1997380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2000883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2004362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2007923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2011489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2014998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2018497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2021997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2025486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2029005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2032502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2035968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2039460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2042971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2046468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2049947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2053538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2057138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2061028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2064533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2068050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2071562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2075064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2078564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2082046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2085531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2089017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2092498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2095989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2099637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2103263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2106725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2110227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2113755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2117234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2120716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2124226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2127758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2131233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2134730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2138246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2141780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2145385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2148975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2152489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2156004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2159916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2163536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2166960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2170467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2173950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2177401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2180906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2184439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2187936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2191556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2195226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2198732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2202215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2205710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2209200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2212717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2216210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2219704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2223207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2226727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2230246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2233746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2237342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2240954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2244462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2247963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2251452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2254945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2258762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2262382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2265880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2269391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2272877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2276377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2279877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2283541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2287166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2290657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2294162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2297684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2301192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2304683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2308194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2311732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2315247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2318759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2322271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2325805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2329392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2332972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2336472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2339968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2343466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2346945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2350437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2353953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2357458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2361355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2364870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2368387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2371885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2375513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2379125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2382640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2386140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2389634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2393123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2396640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2400148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2403649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2407157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2410692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2414206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2417679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2421248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2424825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2428309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2431786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2435282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2438779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2442265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2445748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2449244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2452750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2456242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2460910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2464387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2468011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2472660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2476092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2479615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2483283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2486678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2490161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2493690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2497239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2500748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2504244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2507748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2511229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2514759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2518330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2521880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2525406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2528913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2532400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2535886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2539404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2542912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2546393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2549892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2553424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2556923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2561210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2565079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2569244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2572822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2576228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2579760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2583277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2586797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2590303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 58%] 2024-08-08T21:00:12.2593795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2597305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2600793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2604265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2607819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2611445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2615011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2618496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2621999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2625516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2629018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2632490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2635997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2639501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2643006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2646498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2649990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2653565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2657141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2661162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2664670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2668199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2671709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2675186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2678661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2682156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2685627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2689111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2692674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2696240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2702870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2712186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2721386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2730573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2739706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2748798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2757909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2767477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2776629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2785728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2794853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2804100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2813264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2822556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2831932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2841282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2850416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2859769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2869035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2878159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2887209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2896251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2905369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2914521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2923642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2932735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2941836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2951253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2960872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2970016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2979133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2988361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.2997476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3006584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3015730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3024910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3034056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3043210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3052354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3061901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3071284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3080549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3089634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3098719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3107782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3116807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3125928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3135096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3144139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3153203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3162633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3171869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3180963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3190281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3199626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3208749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3217905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3226994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3236168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3245387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3254552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3263908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3273021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3282206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3291377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3300471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3309909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3319244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3328243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3337276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3346364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3355479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3364896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3374025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3383161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3392307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3401436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3410559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3419714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3429142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3438473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3447551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3456721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3466230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3475404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3484526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3493690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3502874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3511976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3521068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3530131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3539206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3548511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3557760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3567155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3576312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3585446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3594488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3599170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3603859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3608482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3613132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3617805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3622540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3627158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3632036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3636886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3641627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3646260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3650987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3655659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3660616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3665305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3669977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3674578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3679260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3683864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3688447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3693334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3698217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3702836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3707470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3712138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3716818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3721419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3726049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3730763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3735416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3740108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3744768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3749505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3754314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3759404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3764334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3769019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3773787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3778446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3783149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3787782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3792483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3797109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3801778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3806448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3811160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3815929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3820807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3825576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3830287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3834892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3839576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3844230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3848900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3853520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3858159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3863060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3866608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3868421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3870315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3872276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3874157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3875959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3877757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3879571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3881369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3883172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3887576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3894224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3898417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3900223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3902073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3903919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3905759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3907546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3909362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3911186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3912998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3914809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3916600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3918422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3920356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3922268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3924147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3925945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3927783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3929580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3931397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3933187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3935011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3936786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3938590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3940385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3942207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3944051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3945976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3947829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3949653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3951437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3953229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3955043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3956857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3959097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3960966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3962783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3964600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3966395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3968306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3970292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3972169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3973974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3975923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3977742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3979540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3981323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3983120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3984907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3986719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3988496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3990301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3992134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3994067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3995893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3997693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.3999498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4001328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4003107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4004914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4006721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4008553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4010338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4012142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4013967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4015823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4017727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4019568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4021367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4023176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4024969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4026654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4028468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4030284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4032083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4033873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4035685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4037492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4039336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4041217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4043059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4044885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4046665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4048473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4050278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4052108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4053896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4055715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4057522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4059765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4061630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4063546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4065472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4067355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4069129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4070933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4072735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4074544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4076341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4078139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4079951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4081751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4083552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4085348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4087157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4089049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4090964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4092798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4094617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4096431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4098232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4100027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4101846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4103656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4105442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4107250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4109043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4110853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4112666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4114557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4116397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4118209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4119985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4121791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4123579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4125396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4127173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4128963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4130759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4132574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4134383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4136219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4138131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4139989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4141790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4143593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4145417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4147232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4149042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4150837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4152633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4154430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4156223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4158004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4160287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4162263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4164114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4165923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4167730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4169558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4171334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4173143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4174944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4176765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4178544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4180359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4182163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4184052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4185929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4187797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4189593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4191424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4193206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4194997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4196800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4198606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4200391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4202164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4203982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4205788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4207626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4209507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4211368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4213169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4214961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4216752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4218564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4220363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4222159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4223946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4225746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4227572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4229354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4231273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4233167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4235049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4236836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4238643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4240439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4242253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4244029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4245831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4247619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4249439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4251224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4253009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4254807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4256656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4258919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4260871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4262690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4264500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4266296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4268080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4269896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4271700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4273497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4275302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4277120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4278925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4280820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4282697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4284532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4286344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4288128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4289922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4291709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4293532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4295305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4297100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4298886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4300716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4302492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4304375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4306247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4308120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4309891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4311695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4313492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4315289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4317078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4318871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4320678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4322481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4324277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4326068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4327898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4329775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4331601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4333373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4335184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4336978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4338765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4340549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4342338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4344154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4345918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4347711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4349506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4351370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4353249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4355095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4356889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4359078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4360968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4362817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4364595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4366426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 59%] 2024-08-08T21:00:12.4368209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4370016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4371808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4373608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4375388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4377250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4379183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4381049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4382843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4384637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4386474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4388257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4390058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4391839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4393669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4395497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4397319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4399125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4401003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4402923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4404775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4406612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4408510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4410285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4412086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4413923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4415726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4417555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4419352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4421172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4422981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4424851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4426738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4428603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4430407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4432243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4434047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4435878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4437684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4439514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4441321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4443132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4444963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4446798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4448654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4450572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4452510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4454336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4456159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4457969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4460169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4462012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4463816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4465617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4467433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4469267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4471054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4472952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4474880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4476774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4478572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4480403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4482209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4484037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4485827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4487646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4489454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4491390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4493096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4494921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4496768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4498676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4500529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4502334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4504146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4505956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4507750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4509554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4511375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4513183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4514992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4516795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4518617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4520467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4522354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4524197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4526028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4527853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4529656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4531474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4533291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4535133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4536933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4538757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4540575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4542419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4544271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4546182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4548027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4549851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4551639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4553458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4555270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4557108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4559261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4561122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4562947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4564769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4566588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4568476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4570431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4572310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4574147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4575931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4577767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4579600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4581425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4583241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4585065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4586892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4588717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4590531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4592375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4594289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4596117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4597931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4599754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4601598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4603383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4605206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4607020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4608862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4610659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4612485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4614291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4616158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4618034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4619918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4621730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4623557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4625371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4627174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4629003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4630835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4632656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4634464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4636280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4638090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4639929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4641811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4643684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4645495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4647308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4649107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4650923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4652735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4654523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4656345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4658157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4660414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4662229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4664131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4666064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4668980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4670790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4672628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4674534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4676287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4678084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4679917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4681720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4683543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4685330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4687130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4689009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4690921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4692778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4694590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4696477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4698229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4700040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4701845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4713828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4715794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4717625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4719437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4721274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4723186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4725108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4726968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4728792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4730614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4732413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4734236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4736028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4737857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4739627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4741439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4743251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4745084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4746913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4748814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4750658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4752486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4754268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4756091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4757892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4760199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4762030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4763832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4765661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4767480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4769301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4771210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4773161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4775038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4776838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4778641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4780469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4782255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4784046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4785838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4787660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4789470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4791276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4793066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4794867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4796736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4798618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4800481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4802276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4804107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4805895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4807707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4809521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4811360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4813153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4814978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4816782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4818611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4820448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4822353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4824175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4825972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4827770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4829560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4831372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4833192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4834996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4836790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4838609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4840418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4842217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4844054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4845960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4847811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4849611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4851420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4853203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4855034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4856821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4859020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4860894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4862726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4864514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4866329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4868202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4870146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4871982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4873784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4875582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4877412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4879189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4881003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4882800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4884605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4886396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4888189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4890010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4891910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4893818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4895659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4897489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4899316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4901133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4902945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4904773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4906590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4908398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4910200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4912018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4913826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4915645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4917537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4919383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4921219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4923014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4924832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4926628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4928454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4930245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4932060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4933867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4935708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4937523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4939356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4941249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4943133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4944924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4946737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4948558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4950373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4952178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4953986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4955794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4957595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4959759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4961654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4963574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4965516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4967382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4969175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4970994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4972809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4974603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4976397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4978205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4980035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4981822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4983637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4985454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4987344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4989223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4991082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4992885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4994721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4996515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.4998333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5000120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5001941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5003718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5005531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5007338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5009149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5011093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5012873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5014736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5016543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5018341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5020144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5021961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5023774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5025573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5027362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5029179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5030994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5032793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5034630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5036548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5038404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5040187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5041998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5043789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5045603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5047373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5049167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5050966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5052787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5054571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5056378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5058163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5060462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5062380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5064270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5066074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5067911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5069701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5071500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5073352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5075165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5076974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5078781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5080608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5082422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5084283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5086199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5088044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5089840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5091640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5093437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5095259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5097078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5098884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5100679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5102483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5104307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5106091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5107944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5109830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5111703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5113492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5115310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5117119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5118957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5120753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5122582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5124393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5126236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5128031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5129857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5131690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5133578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5135418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5137224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5139052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5140862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5142664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5144459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5146277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5148082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5149886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5151688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5153511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5155361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5157277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5159478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5161364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5163187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5164977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5166791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5168593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5170417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5172198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5174008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5175805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5177615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5179475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5181393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5183249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5185072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5186851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5188665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5190454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5192274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5194055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5195843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5197665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5199480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5201289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5203133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5205029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5206889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5208698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 60%] 2024-08-08T21:00:12.5210506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5212335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5214151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5215956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5217755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5219565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5221359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5223140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5224953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5226806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5228718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5230550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5232359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5234164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5235995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5237781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5239591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5241390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5243210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5244989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5246797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5248599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5250476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5252364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5254253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5256064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5257877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5260061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5261916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5263728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5265534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5267329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5269108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5270918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5272724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5274597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5276509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5278383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5280181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5281977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5283774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5285576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5287397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5289185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5290996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5292801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5294633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5296421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5298239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5300085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5302001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5303832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5305649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5307459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5309254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5311029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5312830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5314633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5316447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5318247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5320040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5321846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5323692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5325570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5327414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5329229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5331039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5332845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5334636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5336456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5338270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5340081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5341886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5343708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5345519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5347356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5349251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5351087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5352910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5354695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5356496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5358290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5360519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5362302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5364109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5365905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5367727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5369508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5371397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5373320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5375198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5376969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5378765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5380580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5382385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5384178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5385971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5387774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5389580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5391389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5393174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5395015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5396888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5398723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5400497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5402305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5404109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5405879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5407676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5409463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5411280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5413052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5414844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5416727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5418516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5420374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5422239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5424035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5425859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5427644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5429461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5431262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5433095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5434881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5436675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5438487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5440281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5442118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5443975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5445830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5447635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5449432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5451223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5453032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5454836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5456631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5458773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5460666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5462600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5464539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5466447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5468424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5470326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5472111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5473925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5475731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5477612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5479362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5481180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5482971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5484790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5486572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5488446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5490166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5492041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5493916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5495775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5497566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5499375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5501168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5502962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5504781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5506595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5508391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5510181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5512000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5513807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5515653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5517554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5519426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5521243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5523059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5524855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5526674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5528467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5530248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5532050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5533843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5535668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5537453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5539304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5541183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5543046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5544829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5546643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5548437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5550258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5552041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5553849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5555639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5557448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5559617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5561538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5563433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5565366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5567225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5569018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5570821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5572612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5574390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5576161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5577973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5579771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5581561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5583346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5585154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5587002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5588859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5590706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5592497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5594327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5596108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5597916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5599707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5601537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5603318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5605129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5606934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5608764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5610589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5612496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5614325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5616138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5617910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5619694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5621505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5623312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5625115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5626905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5628710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5630517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5632314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5634149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5636099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5637953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5639748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5641532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5643352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5645166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5646971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5648769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5650583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5652399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5654185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5655997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5657829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5660145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5662019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5663815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5665609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5667438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5669216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5671026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5672826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5674647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5676425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5678237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5680030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5681826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5683681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5685566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5687424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5689233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5691033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5692828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5694637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5696452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5698250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5700038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5701842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5703639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5705428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5707261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5709139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5711002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5712784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5714589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5716382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5718200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5719977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5721784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5723574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5725394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5727178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5728987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5730828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5732741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5734581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5736398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5738209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5740023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5741832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5743631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5745448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5747247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5749041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5750826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5752640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5754497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5756384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5758226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5760452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5762267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5764066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5765855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5767665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5769470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5771245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5773056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5774860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5776683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5778542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5780474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5782344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5784168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5785952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5787766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5789531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5791344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5793113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5794909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5796697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5798503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5800290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5802111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5803998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5805846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5807633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5809412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5811231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5813029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5814816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5816626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5818419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5820219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5822017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5823815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5825624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5827473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5829329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5831179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5832965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5834782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5836553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5838344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5840134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5841951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5843724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5845526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5847310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5849121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5850946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5852829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5854662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5856480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5858263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5860442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5862304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5864133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5865917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5867707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5869516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5871331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5873128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5875001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5876950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5878808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5880601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5882385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5884197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5885992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5887791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5889574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5891362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5893186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5894965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5896767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5898602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5901071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5902883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5904684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5906483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5908303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5910079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5911880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5913671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5915493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5917379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5919057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5920855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5922689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5924560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5926378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5928185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5929981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5931777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5933557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5935359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5937164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5938955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5940732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5942550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5944355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5946190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5948054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5949897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5951723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5953500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5955314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5957112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5959309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5961163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5962971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5964758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5966572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5968343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5970134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5972008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5973971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5975811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5977600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5979408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5981214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5983003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5984788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5986613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5988430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5990237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5992036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5993858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5995721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5997616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.5999469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6001297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6003119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6004929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6006732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6008528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6010348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6012125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6013939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6015744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6017568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6019397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6021343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6023183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6025031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6026806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6028619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6030422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6032331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6034035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6035846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6037658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 61%] 2024-08-08T21:00:12.6039484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6041298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6043145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6045054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6046960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6048734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6050541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6052447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6054154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6055952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6057746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6059921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6061788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6063594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6065396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6067311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6069256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6071107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6072922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6074726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6076657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6078348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6080202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6082013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6083848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6085636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6087456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6089257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6091139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6093014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6094871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6096661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6098484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6100264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6102045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6103864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6105682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6107484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6109270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6111084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6112897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6114733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6116620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6118487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6120314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6122117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6123915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6125741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6127562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6129387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6131191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6132998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6134838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6136637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6138498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6140381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6142263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6144035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6145844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6147654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6149573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6151272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6153090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6154893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6156718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6158835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6160763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6162658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6164600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6166466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6168260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6170087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6171910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6173719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6175534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6177354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6179221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6181039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6182888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6184704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6186541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6188419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6190260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6192082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6193892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6195784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6197536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6199295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6201120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6202911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6204723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6206521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6208346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6210120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6211967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6213845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6215780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6217555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6219342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6221239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6223057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6224869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6226663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6228470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6230272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6231980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6233777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6235689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6237630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6239433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6241263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6243060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6244772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6246564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6248352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6250176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6251982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6253785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6255583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6257385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6259652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6261648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6263547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6265357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6267202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6268998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6270822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6272638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6274435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6276206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6278138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6281568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6285070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6288620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6292206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6295823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6299276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6302853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6306238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6309714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6313220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6316705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6320176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6323673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6327190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6330681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6334192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6337778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6341328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6344825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6348364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6351793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6355257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6359138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6362658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6366136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6369630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6373111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6376572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6380115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6383819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6387295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6390763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6394250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6397755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6401216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6404784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6408179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6411683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6415172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6418648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6422194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6425682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6429278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6432840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6436260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6439743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6443216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6446658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6450110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6453602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6457075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6460979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6464475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6467976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6471526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6475136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6478681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6482178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6485662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6489134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6492618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6496111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6499599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6503082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6506572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6510086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6513579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6517171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6520723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6524309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6527810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6531279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6534656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6538278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6541615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6545092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6548565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6552045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6555534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6559354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6562911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6566493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6570110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6573639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6577115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6580620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6584104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6587580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6591042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6594647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6598018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6601530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6604959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6608424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6611918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6615498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6619054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6622452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6625973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6629372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6632841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6636314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6639786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6643226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6646701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6650201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6653681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6657204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6661250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6664810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6668307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6671788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6675288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6678799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6682281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6685756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6689234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6692727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6696198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6699657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6703205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6706797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6710345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6713809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6717287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6720776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6724266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6727728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6731204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6734710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6738183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6741655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6745147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6748709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6752268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6755797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6759671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6763291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6766787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6770252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6773730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6777197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6780663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6784109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6787581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6791077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6794639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6798234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6801755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6805252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6808720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6812174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6815651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6819150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6822621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6826088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6829579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6833099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6836580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6840114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6843699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6847241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6850730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6854209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6857690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6861595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6865083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6868527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6872011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6875503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6878985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6882455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6886010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6889653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6893180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6896656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6900132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6903629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6907099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6910557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6914043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6917515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6921001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6924483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6927965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6931463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6935017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6938565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6942084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6945589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6949037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6952481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6955944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6960269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6963784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6967239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6970720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6974209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6977666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6981230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6984848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6988397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6991887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6995362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.6998851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7002360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7005855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7009311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7012803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7016358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7019827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7023267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7026896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7030470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7033966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7037419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7040904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7044402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7047883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7051344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7054724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7058198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7062140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7065627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7069104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7072696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7076326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7079838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7083330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7086830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7090328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7093787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7097277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7100781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7104254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7107731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7111223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7114702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7118214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7121765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7125288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7128762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7132240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7135709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7139181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7142681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7146161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7149608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7153085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7156597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7160455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7164023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7167640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7171204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7174672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7178152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7181646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7185154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7188645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7192118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7195578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7199058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7202528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7205978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7209480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7213062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7216621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7220060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7223526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7227021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7230503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7233960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7237457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7240959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7244421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7247888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7251378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7254886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7258774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7262533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7266061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7269565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7273081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7276557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7280037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7283517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7287006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7290422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7293907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7297411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7300898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7304393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7307964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7311509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7314964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7318432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7321899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7325375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7328844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7332300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7335761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7339260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7342740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7346213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7349742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7353335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7356861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7360711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7364242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7367729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7371188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7374640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7378105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7381573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7385045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7388503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7391960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7395541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7399131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7402667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7406125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7409621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 62%] 2024-08-08T21:00:12.7413106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7416558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7420039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7423534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7427018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7430466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7433952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7437448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7440980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7444534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7448045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7451500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7454968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7458761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7462300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7465792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7469271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7472734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7476186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7479721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7483148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7486681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7490290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7493857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7497353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7500749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7504236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7507743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7511233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7514714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7518234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7521690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7525173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7528640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7532167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7535796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7539253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7542690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7546145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7549624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7553090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7556548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7560442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7563943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7567398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7570849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7574328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7577792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7581341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7584935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7588461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7591934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7595413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7598871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7602344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7605838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7609312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7612763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7616195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7619661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7623106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7626581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7630129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7633645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7637074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7640519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7643979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7647443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7650899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7654351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7657818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7661739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7665229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7668691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7672292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7675979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7679549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7683005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7686516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7690002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7693465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7696930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7700410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7703891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7707329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7710774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7714245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7717791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7721344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7724861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7728324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7731798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7735266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7738730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7742219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7745739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7749242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7752710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7756226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7760111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7763772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7767412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7770976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7774493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7777974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7781467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7784947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7788444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7791925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7795405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7798894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7802381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7805868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7809417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7812974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7816532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7820012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7823482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7826987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7830490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7833983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7837457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7840958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7844496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7847959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7851446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7854940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7858889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7863476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7866965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7870449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7873937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7877412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7880882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7884376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7887886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7891374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7894826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7898321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7901814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7905345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7908922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7912449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7915956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7919414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7922879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7926376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7929882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7933378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7936856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7940407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7943844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7947333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7950870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7954489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7958027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7961961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7965415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7968892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7972382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7975853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7979316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7982785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7986283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7989748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7993219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.7996807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8000448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8004004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8007479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8010979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8014485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8017997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8021609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8024990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8028505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8031999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8035464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8038953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8040825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8042695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8044572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8046364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8048204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8050024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8051800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8053653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8055442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8057218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8059391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8061269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8063099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8064989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8066911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8068819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8070641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8072463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8074275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8076095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8077918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8079736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8081548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8083366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8085177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8086982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8088833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8090745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8092603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8094396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8096221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8098056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8099862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8101651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8103475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8105284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8107115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8108904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8110721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8112566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8125028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8127044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8128897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8130727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8132571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8134365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8136182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8137983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8139787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8141590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8143375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8145191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8147049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8148962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8150785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8152593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8154401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8156203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8158020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8160250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8162147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8163958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8165755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8167588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8169412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8171314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8173272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8175144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8176980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8178771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8180602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8182397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8184216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8185993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8187803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8189603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8191432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8193228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8195083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8196970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8198842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8200624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8202428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8204249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8206065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8207870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8209669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8211490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8213304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8215109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8216907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8218727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8220599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8222513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8224334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8226139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8227942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8229748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8231522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8233350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8235143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8236923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8238731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8240529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8242350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8244176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8246067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8247909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8249729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8251511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8253319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8255121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8256960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8259096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8260968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8262840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8264660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8266472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8268356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8270298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8272184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8273957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8275748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8277563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8279370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8281170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8282963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8284773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8286576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8288378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8290173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8292019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8293931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8295777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8297591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8299400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8301236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8303029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8304854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8306659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8308491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8310302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8312095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8313886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8315744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8317619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8319452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8321251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8323062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8324854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8326636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8328447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8330260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8332141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8333856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8335672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8337473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8339310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8341180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8343032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8344841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8346647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8348443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8350256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8352063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8353854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8355663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8357455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8359641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8361475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8363362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8365276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8367155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8368920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8370717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8372508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8374323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8376098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8377899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8379700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8381513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8383319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8385117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8386983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8388925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8390752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8392547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8394365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8396203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8397993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8399784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8401591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8403388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8405175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8406959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8408778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8410580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8412403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8414352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8416201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8418016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8419794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8421603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8423407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8425240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8427020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8428822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8430624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8432455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8434237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8436094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8437983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8439886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8441657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8443510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8445265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8447066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8448878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8450643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8452448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8454254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8456053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8457845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8460218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8462247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8464112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8465900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8467720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8469528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8471350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8473122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8474928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8476752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8478537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8480378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8482159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8484029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8486004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8487786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8489573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8491387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8493159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8494951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8496770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8498582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8500364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8502176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8503969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8505771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8507623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8509481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8511342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8513141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8514938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8516725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8518533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8520336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8522136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8523936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8525751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8527560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8529359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8531218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8533091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8534926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8536695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8538491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8540291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8542117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8543883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8545684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8547478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8549302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8551077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8552871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 63%] 2024-08-08T21:00:12.8554705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8556616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8558794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8560628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8562475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8564317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8566104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8567904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8569733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8571547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8573354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8575155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8576964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8578756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8580643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8582559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8584442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8586267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8588056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8589847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8591659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8593466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8595245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8597056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8598853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8600674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8602454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8604302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8606191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8608069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8609855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8611669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8613495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8615300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8617091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8618902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8620690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8622509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8624283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8626072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8627929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8629824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8631667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8633459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8635271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8637073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8638863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8640660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8642496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8644294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8646079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8647863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8649685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8651544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8653426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8655284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8657084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8659267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8661115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8662928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8664718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8666530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8668292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8670081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8671869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8673686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8675545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8677470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8679321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8681137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8682911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8684691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8686501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8688315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8690118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8691910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8693726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8695556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8697345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8699182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8701087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8702949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8704749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8706545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8708348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8710139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8711927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8713710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8715510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8717335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8719120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8720931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8722765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8724675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8726502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8728305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8730098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8731921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8733699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8735505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8737298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8739125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8740904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8742715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8744514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8746390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8748261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8750098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8751898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8753695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8755486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8757265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8759455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8761329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8763120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8764906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8766715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8768517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8770305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8772172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8774106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8775967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8777745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8779557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8781358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8783180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8784964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8786784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8788578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8790399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8792184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8793992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8795820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8797775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8799589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8801387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8803185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8804994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8806787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8808577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8810386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8812187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8813978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8815770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8817584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8819427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8821306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8823137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8824951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8826784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8828573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8830363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8832182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8833996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8835781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8837588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8839382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8841205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8843025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8844919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8846755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8848569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8850351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8852163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8853944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8855763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8857545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8859698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8861577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8863404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8865189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8867060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8868995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8870867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8872654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8874438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8876249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8878047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8879846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8881633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8883429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8885215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8886997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8888769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8890571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8892413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8894268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8896112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8897892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8899715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8901486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8903280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8905075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8906895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8908664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8910465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8912261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8914083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8915906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8917806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8919644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8921469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8923258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8925046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8926842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8928756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8930427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8932197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8933998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8935800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8937595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8939545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8941328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8943175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8944965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8946751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8948558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8950463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8952125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8953923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8955717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8957536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8959717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8961550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8963430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8965375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8967211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8969015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8970799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8972613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8974366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8976181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8977941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8979758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8981530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8983311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8985120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8986960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8988833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8990658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8992453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8994254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8996048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8997833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.8999634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9001469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9003243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9005023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9006840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9008648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9010493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9012364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9014208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9016009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9017773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9019564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9021351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9023164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9024938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9026739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9028524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9030337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9032108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9033896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9035716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9037633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9039423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9041203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9043009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9044819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9046604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9048388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9050196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9051986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9053769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9055553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9057349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9059572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9061539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9063374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9065174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9066963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9068748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9070517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9072316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9074104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9075869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9077655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9079446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9081260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9083075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9084958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9086792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9088635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9090406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9092201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9093986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9095822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9097583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9099368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9101163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9102949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9104737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9106549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9108454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9110315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9112081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9113858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9115664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9117455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9119240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9121022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9122823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9124619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9126407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9128188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9129976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9131843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9133711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9135555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9137349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9139196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9140955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9142746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9144524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9146335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9148092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9149915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9151658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9153472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9155278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9157141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9159599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9161506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9163313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9165091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9166887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9168684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9170470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9172237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9174036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9175835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9177622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9179575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9182129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9183994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9185788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9187579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9189360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9191162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9192914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9194697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9196479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9198287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9200061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9201850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9203672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9205583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9207396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9209191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9211011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9212810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9214581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9216368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9218180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9219988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9221783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9223570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9225377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9227224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9229097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9230928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9232723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9234516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9236307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9238077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9239884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9241686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9243473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9245250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9247044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9248867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9250628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9252463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9254336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9256185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9257968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9260114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9261973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9263794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9265570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9267367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9269169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9270958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9272732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9274534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9276387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9278300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9280137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9281901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9283700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9285571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9287271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9289041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9290832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9292623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9294405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9296176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9297974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9299821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9301696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9303510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9305292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9307104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9308875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9310669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9312456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9314290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9316061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9317844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9319618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9321421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9323215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9325087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9326912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9328713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9330479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9332253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9334044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9335830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9337616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9339387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9341193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9342972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9344764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9346530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9348371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9350260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9352091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9353867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9355672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9357468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9359604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9361470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9363252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9365056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9366815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9368596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9370374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9372257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9374140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9376005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9377777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 64%] 2024-08-08T21:00:12.9379596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9381361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9383146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9384924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9386737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9388504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9390267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9392064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9393862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9395706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9397577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9399430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9401215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9402997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9404775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9406555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9408322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9410097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9411857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9413649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9415436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9417204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9418991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9420810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9422710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9424524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9426312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9428100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9429927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9431678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9433474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9435262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9437080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9438881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9440661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9442444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9444301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9446148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9447974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9449768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9451557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9453333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9455118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9456898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9459050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9460886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9462696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9464496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9466292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9468171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9470070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9471955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9473768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9475556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9477369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9479173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9481002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9482803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9484634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9486415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9488247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9490036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9491912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9493770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9495628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9497402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9499199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9500993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9502819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9504599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9506388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9508200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9510004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9511796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9513591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9515455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9517348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9519186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9520969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9522783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9524597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9526401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9528198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9530023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9531833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9533639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9535434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9537223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9539075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9540940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9542835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9544649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9546482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9548244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9550052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9551840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9553664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9555438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9557238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9559395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9561297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9563062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9564934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9566897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9568764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9570564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9572370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9574186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9576004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9577805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9579601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9581403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9583198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9585003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9586770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9588618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9590501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9592347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9594157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9595929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9597750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9599516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9601317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9603119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9604946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9606731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9608541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9610343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9612220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9614090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9615955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9617779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9619605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9621389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9623205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9625002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9626807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9628600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9630392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9632226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9634017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9635855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9637732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9639585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9641392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9643200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9644997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9646806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9648613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9650429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9652210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9654034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9655865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9657649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9659907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9661915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9663805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9665595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9667411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9669201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9671012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9672783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9674584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9676408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9678204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9679984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9681797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9683630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9685543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9687367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9689155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9690968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9692768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9694556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9696339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9698148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9699947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9701734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9703527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9705355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9707198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9709080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9710909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9712709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9714505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9716277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9718080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9719875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9721683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9723449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9725239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9727027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9728850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9730622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9732465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9734343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9736227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9738013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9739818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9741621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9743450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9745239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9747040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9748863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9750681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9752490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9754281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9756128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9758032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9760167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9762040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9763860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9765668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9767462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9769256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9771069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9772897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9774670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9776462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9778249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9780147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9782052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9783924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9785726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9787557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9789336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9791147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9792956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9794784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9796557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9798360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9800153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9801969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9803783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9805667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9807505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9809334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9811115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9812902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9814721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9816507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9818301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9820092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9821904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9823710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9825508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9827336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9829251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9831104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9832906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9834711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9836510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9838328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9840107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9841915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9843701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9845519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9847289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9849081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9850920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9852824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9854642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9856436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9858223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9860406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9862224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9864024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9865816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9867618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9869434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9871201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9873012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9874895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9876817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9878673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9880483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9882290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9884090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9885878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9887681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9889478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9891267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9893043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9894852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9896658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9898430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9900272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9902148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9904013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9905785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9907583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9909372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9911183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9913065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9914746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9916533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9918355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9920154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9921930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9923767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9925709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9927553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9929335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9931131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9932921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9934723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9936494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9938296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9940084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9941865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9943645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9945440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9947280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9949146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9950963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9952802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9954574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9956352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9958205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9960338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9962215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9964004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9965814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9967611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9969433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9971297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9973232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9975081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9976892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9978670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9980493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9982259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9984079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9985863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9987650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9989451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9991256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9993047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9994875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9996774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:12.9998633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0000423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0002214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0004034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0005840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0007638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0009436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0011264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0013070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0014851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0016658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0018499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0020430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0022231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0024029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0025826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0027653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0029428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0031231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0033024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0034847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0036624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0038431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0040219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0042036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0043847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0045722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0047579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0049392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0051196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0052996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0054812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0056622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0058760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0060563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0062440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0064243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0066030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0067884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0069821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0071695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0073492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0075282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0077074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0078893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0080668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0082469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0084258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0086072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0087857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0089655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0091496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0093401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0095241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0097042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0098835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0100664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0102433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0104211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0106011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0107804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0109584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0111367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0113168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0115002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0116872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0118694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0120500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0122326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0124094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0125867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0127678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0129483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0131272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0133059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0134856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0136680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0138506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0140402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0142246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0144076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0145877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0147664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0149447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0151270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0153034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0154828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0156617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0158762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0160581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0162395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0164294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0166226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0168085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0169871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0171762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0173490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0175283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0177079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0178894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0180706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0182515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0184311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0186123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0187986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0189896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0191731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0193521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0195336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0197108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0198905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0200703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0202524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0204292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 65%] 2024-08-08T21:00:13.0206090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0207885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0209705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0211530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0213417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0215252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0217067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0218843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0220642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0222436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0224247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0226037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0227820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0229626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0231438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0233240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0235078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0236959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0238798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0240586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0242361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0244169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0245969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0247757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0249545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0251333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0253150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0254935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0256723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0258935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0260983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0262842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0264648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0266437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0268261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0270044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0271856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0273653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0275483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0277281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0279073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0280857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0282696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0284618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0286418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0288221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0290018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0291811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0293597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0295409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0297206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0298998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0300776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0302587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0304381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0306162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0307987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0309942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0311794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0313592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0315398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0317165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0318981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0320754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0322557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0324333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0326138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0327911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0329697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0331517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0333416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0335228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0337024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0338811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0340609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0342411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0344166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0345972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0347763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0349541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0351324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0353132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0354981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0356864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0359053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0360900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0362787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0364584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0366368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0368165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0369963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0371729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0373515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0375304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0377114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0378999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0380906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0383336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0385149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0386920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0388718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0390507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0392322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0394101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0395902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0397694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0399506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0401301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0403127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0405028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0406883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0408669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0410453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0412251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0414043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0415820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0417599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0419405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0421219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0422982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0424760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0426555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0428396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0430252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0432092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0433879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0435689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0437452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0439243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0441030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0442874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0444619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0446403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0448189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0450001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0451833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0453691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0455551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0457316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0459456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0461286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0463092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0464887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0466671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0468438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0470224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0472011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0473792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0475642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0477572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0479431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0481215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0483000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0484807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0486631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0488377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0490182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0492058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0493796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0495571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0497375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0499195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0501083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0502916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0504678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0506464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0508281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0510046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0511844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0513626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0515438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0517216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0518985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0520791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0522580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0524408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0526278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0528162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0529939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0531739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0533521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0535324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0537130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0538916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0540698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0542497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0544288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0546040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0547862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0549746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0551604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0553366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0555164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0556948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0559106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0560912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0562749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0564533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0566343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0568117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0569920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0571789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0573742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0575584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0577372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0579186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0580992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0582781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0584562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0586360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0588156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0589952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0591705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0593513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0595348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0597216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0599043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0600841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0602645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0604416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0606208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0607990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0609805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0611578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0613370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0615161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0616977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0618788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0620672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0622502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0624318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0626093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0627894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0629665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0631474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0633238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0635012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0636810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0638601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0640388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0642162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0644002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0645897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0647727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0649495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0651308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0653101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0654888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0656674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0658804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0660649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0662494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0664304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0666098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0668000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0669900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0671770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0673549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0675356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0677119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0678907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0680697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0682507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0684265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0686062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0687911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0689716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0691610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0693538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0695732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0697581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0699374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0701170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0702986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0704803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0706610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0708408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0710227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0712044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0713853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0715708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0717606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0719461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0721257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0723045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0724856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0726688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0728467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0730282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0732079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0733907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0735692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0737502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0739357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0741270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0743097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0744906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0746703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0748536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0750329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0752143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0753945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0755758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0757559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0759723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0761591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0763398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0765299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0767177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0769053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0770862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0772661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0774446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0776262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0778394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0780146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0781936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0783764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0785582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0787398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0789364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0791269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0793043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0794850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0796687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0798522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0800378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0802073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0803892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0805685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0807507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0809286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0811095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0812950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0814859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0816690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0818505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0820429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0822141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0823922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0825708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0827538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0829349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0831145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0833033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0834760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0836648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0838509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0840352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0842210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0843974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0845784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0847582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0849387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0851192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0852973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0854770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0856567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0858772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0860534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0862464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0864408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0866291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0868074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0869882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0871683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0873507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0875321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0877108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0878907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0880724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0882538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0884337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0886201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0888102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0889953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0891774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0893595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0895384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0897172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0898950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0900770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0902564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0904371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0906163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0907972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0909822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0911701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0913554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0915354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0917183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0918974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0920780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0922585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0924423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0926210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0928025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0929835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0931665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0933545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0935447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0937279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0939082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0940881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0942678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0944521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0946301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0948109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0949892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0951699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0953529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0955311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0957097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0959362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0961356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0963272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0965060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0966875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0968687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0970464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0972270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0974064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0975882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0977660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0979466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0981254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0983108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0984961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0986800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0988580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0990395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0992171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0994016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0995760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0997607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.0999375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1001122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1002936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1004743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1006590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1008476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1010338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1012150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1013951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1015742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1017559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1019372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1021179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1022975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1024766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1026585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1028359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1030203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 66%] 2024-08-08T21:00:13.1032080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1033943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1035718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1037520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1039308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1041128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1042901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1044714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1046552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1048420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1050238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1052087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1053922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1055809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1057733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1060010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1061936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1063793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1065640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1067530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1069309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1071163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1072971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1074796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1076643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1078482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1080395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1082341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1084255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1086095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1087917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1089767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1091620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1093466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1095285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1097139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1098974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1100844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1102662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1104567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1106487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1108450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1110290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1112082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1113908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1115765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1117583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1119419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1121250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1123093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1124928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1126749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1128636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1130557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1132434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1134264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1136113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1137953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1139779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1141607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1143453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1145296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1147137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1148971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1150814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1152697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1154597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1156486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1158310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1160549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1162396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1164229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1166061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1167916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1169719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1171555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1173382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1175226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1177116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1179079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1180991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1182854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1184672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1186506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1188361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1190214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1192052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1193876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1195728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1197574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1199412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1201332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1203201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1205103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1206906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1208727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1210573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1212416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1214250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1216071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1217892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1219747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1221562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1223395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1225264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1227206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1229069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1230909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1232748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1234730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1236514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1238357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1240227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1242070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1243906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1245726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1247553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1249428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1251362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1253224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1255044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1256902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1259057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1260937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1262836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1264698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1266508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1268349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1270173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1272012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1273899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1275872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1277765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1279623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1281444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1283291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1285116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1286972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1288905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1290626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1294250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1297841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1301348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1304842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1308445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1312111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1315532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1319107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1322551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1326080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1329596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1333186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1336622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1340210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1344115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1347636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1351234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1354887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1358818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1362489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1366044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1369703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1373119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1376750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1380150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1383770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1387201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1390672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1394171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1397802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1401444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1405019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1408548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1412162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1415695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1419187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1422721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1426236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1429753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1433121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1436636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1440431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1443966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1447703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1451378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1454837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1458718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1509475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1513242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1516759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1520301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1523832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1527339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1530959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1534425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1538105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1541805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1545436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1548959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1552465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1556023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1561611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1565248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1568769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1572341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1575873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1579406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1582941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1586608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1590301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1593917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1597439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1600979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1604502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1608017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1611631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1615056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1618711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1622131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1625634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1629167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1632760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1636426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1639990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1643526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1647087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1650597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1654171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1657686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1661729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1665280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1668823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1672379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1675920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1679590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1683256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1686851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1690394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1693912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1697402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1701024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1704577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1708001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1711614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1715218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1718653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1722173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1725738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1729360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1732949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1736463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1739966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1743496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1747028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1750564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1754099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1757631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1761640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1765186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1768708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1772303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1775964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1779541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1783050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1786581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1790118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1793633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1797134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1800749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1804191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1807737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1811256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1814790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1818375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1822010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1825594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1829123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1832694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1836234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1839765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1843290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1846860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1850406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1853930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1857473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1861515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1865177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1868795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1872400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1875953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1879489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1883019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1886541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1890073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1893606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1897128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1900656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1904219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1907759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1911404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1915041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1918640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1922203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1925741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1929293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1932861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1936385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1939917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1943437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1946981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1950509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1954005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1957581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1962638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1966251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1969773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1973295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1976838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1980356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1983872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1987410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1990953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1994490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.1998021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2001565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2005189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2008897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2012457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2015989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2019557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2023099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2026623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2030156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2033696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2037215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2040706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2044238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2047787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2051368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2054981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2058899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2062524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2066075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2069594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2073115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2076677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2080189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2083701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2087217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2090775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2094314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2097910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2101588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2105209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2108734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2112243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2115795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2119294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2122804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2126317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2129852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2133361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2136886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2140394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2143991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2147592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2151164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2154672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2158190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2162207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2165758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2169278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2172841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2176404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2179955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2183503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2187055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2190679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2194377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2197979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2201522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2205032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2208557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2212070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2215605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2219167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2222707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2226218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2229737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2233288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2236880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2240485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2244086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2247632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2251164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2254672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2258223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2262270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2265846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2269382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2272935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2276555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2280043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2283676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2287344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2290950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2294470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2297987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2301512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2305069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2308604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2312127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2315669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2319215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2322765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2326271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2329802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2333393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2337010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2340565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2344104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2347670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2351192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2354723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2358262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2362713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2366282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2369807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2373326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2376852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2380523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2384184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2387779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2391325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2394852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2398363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2401885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2405413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2408941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2412447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2415975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2419541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2423048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2426618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2430249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2433865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2437422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2440965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2444500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2448073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2451679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2455218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2459174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 67%] 2024-08-08T21:00:13.2462769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2466293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2469788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2473426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2477116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2480716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2484240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2487796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2491335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2494851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2498381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2501910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2505453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2508988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2512506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2516048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2519642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2523295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2526884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2530424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2533997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2537527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2541031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2544553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2548077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2551634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2555144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2559058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2562646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2566251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2569903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2573497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2577040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2580566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2584084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2587613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2591146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2594678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2598185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2601721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2605290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2608839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2612394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2616035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2619629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2623156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2626678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2630199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2633731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2637225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2640740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2644265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2647811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2651344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2654904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2658896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2662636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2666248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2669772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2673311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2676864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2680384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2683885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2687426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2690986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2694530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2698130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2701610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2705212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2708839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2712413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2715931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2719466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2722981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2726491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2730018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2733555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2737079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2740593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2744133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2747678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2751249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2754886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2758787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2762378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2765895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2769413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2772935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2776482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2779988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2783510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2787042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2790589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2794128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2797721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2801380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2804941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2808444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2811929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2815443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2818989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2822501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2825981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2829499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2833030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2836545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2840051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2843634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2847280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2850832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2854398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2857934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2861906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2865457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2868988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2872535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2876073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2879612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2883138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2886663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2890289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2893938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2897487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2901007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2904552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2908083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2911596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2915131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2918683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2922185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2925706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2929255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2932820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2936366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2939952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2943614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2947234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2950795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2954406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2957980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2962051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2965636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2969192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2972722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2976279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2979891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2983340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2986981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2990687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2994285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.2997824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3001380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3004954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3008500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3012045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3015608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3019158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3022715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3026250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3029801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3033436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3037090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3040706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3044255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3047838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3051399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3054995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3058918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3062524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3066073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3069581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3073134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3076700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3080349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3084025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3087641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3091191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3094752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3098293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3101842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3105406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3108961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3112494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3116021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3119589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3123139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3126736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3130386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3132280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3134120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3135955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3137791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3139618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3141451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3143256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3145105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3146975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3148778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3150592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3152504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3154455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3156312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3158159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3160383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3162267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3164075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3165924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3167760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3169627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3171446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3173298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3175126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3177077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3179039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3180956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3182780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3184641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3186454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3188279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3190129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3191967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3193793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3195620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3197468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3199308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3201181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3203093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3204986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3206821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3208652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3210475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3212334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3214191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3216028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3217858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3219695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3221562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3223379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3225268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3227176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3229077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3230888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3232732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3234551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3236405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3238221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3240059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3241883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3243740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3245551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3247390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3249263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3251189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3253105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3254931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3256782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3259102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3260909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3262747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3264603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3266446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3268280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3270102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3271931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3273844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3275811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3277698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3279546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3281388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3283199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3285035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3286858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3288716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3290526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3292370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3294208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3296072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3297934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3299873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3302503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3304371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3306196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3308047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3309881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3311749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3313577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3315412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3317259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3319092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3320916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3322782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3324716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3326606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3328436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3330270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3332121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3333962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3335797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3337622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3339466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3341354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3343136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3344954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3346833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3348782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3350648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3352542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3354377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3356235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3358048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3360268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3362103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3363942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3365749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3367582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3369402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3371352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3373278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3375176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3376996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3378859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3380664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3382482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3384331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3386167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3387994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3389815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3391671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3393510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3395425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3397362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3399266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3401108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3402943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3404769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3406602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3408424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3410242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3412049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3413873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3415725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3417535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3419409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3421314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3423211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3425017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3426855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3428679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3430535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3432341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3434183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3436005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3437872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3439688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3441533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3443405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3445326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3447217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3449038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3450877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3452742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3454558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3456378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3458219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3460418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3462243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3464063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3465910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3467825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3469778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3471657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3473501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3475337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3477137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3478974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3480802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3482657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3484463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3486301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3488126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3489978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3491826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3493751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3495601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3497442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3499241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3501073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3502893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3504746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3506551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3508360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3510194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3512023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3513841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3515698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3517633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3519522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3521356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3523181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3525031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3526872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3528706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3530540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3532395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3534236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3536077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3537906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3539771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3541719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3543572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3545407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3547240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3549095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3550895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3552774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3554603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3556455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3558267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3560520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3562351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3564291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3566223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3568122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3569948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3571800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3573637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3575454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3577298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3579143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3580974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3582790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3584621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3586446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3588304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3590203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3592091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3593921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3595746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3597569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3599400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3601229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3603036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3604871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3606695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3608545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3610357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3612237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3614150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3616057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3617872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3619717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3621540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3623394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3625206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 68%] 2024-08-08T21:00:13.3627049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3628860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3630702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3632505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3634324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3636213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3638146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3640013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3641830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3643668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3645496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3647312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3649200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3650975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3652849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3654673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3656490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3658330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3660638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3662598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3664485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3666307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3668166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3669981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3671821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3673634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3675481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3677275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3679100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3680922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3682774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3684625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3686549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3688414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3690256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3692058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3693894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3695723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3697558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3699395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3701217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3703070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3704918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3706754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3708628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3710563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3712451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3714282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3716112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3717950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3719778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3721594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3723414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3725262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3727092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3728900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3730739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3732614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3734548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3736403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3738239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3740061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3741918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3743722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3745554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3747389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3749250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3751075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3752969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3754811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3756722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3759013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3760960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3762772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3764595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3766420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3768232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3770071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3771902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3773728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3775547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3777386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3779218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3781042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3782919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3784863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3786740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3788567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3790385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3792224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3794060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3795874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3797717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3799538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3801394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3803201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3805029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3806877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3808798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3810661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3812489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3814315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3816161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3817958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3819791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3821606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3823429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3825249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3827064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3828908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3830786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3832696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3834568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3836415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3838263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3840102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3841925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3843776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3845619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3847449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3849272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3851109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3852973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3854843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3856745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3859036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3860939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3862750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3864583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3866408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3868257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3870067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3871907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3873728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3875576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3877373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3879296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3881246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3883172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3884984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3886831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3888660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3890496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3892330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3894149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3895969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3897782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3899603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3901412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3903294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3905228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3907101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3908911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3910753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3912582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3914406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3916224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3918068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3919895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3921696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3923533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3925360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3927261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3929164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3931053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3932878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3934731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3936543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3938385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3940201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3942041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3943832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3945660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3947484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3949339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3951182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3953135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3955023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3956850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3958990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3960844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3962676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3964502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3966322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3968135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3969980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3971816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3973643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3975709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3977688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3979587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3981426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3983256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3985066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3986893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3988691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3990527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3992348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3994196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3996008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3997837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.3999696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4001644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4003489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4005324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4007152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4008995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4010793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4012599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4014438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4016265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4018085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4019906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4021740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4023610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4025511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4027370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4029189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4031003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4032820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4034619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4036450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4038277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4040091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4041887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4043703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4045548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4047387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4049297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4051168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4053063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4054870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4056710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4058903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4060801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4062614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4064456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4066276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4068129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4069935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4071862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4073801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4075681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4077492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4079297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4081142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4082977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4084803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4086615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4088448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4090271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4092092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4093902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4095732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4097618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4099526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4101380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4103219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4105050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4106850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4108678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4110504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4112353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4114152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4115988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4117792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4119616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4121455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4123359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4125214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4127055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4128853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4130679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4132485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4134325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4136125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4137929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4139758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4141588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4143401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4145241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4147167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4149043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4150860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4152711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4154549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4156372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4158197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4160416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4162248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4164065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4165883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4167687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4169579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4171547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4173429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4175259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4177070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4178909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4180702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4182528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4184337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4186174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4187967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4189792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4191608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4193502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4195386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4197270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4199092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4200923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4202740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4204542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4206364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4208186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4209996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4211796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4213625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4215445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4217294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4219177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4221046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4222869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4224693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4226492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4228307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4230149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4231951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4233782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4235599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4237456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4239255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4241133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4243034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4244925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4246718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4248550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4250370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4252246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4254040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4255861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4257670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4259853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4261703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4263520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4265433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4267390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4269262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4271070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4272902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4274725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4276544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4278347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4280186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4282010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4283827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4285643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4287480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4289307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4291153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4293069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4294919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4296754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4298549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4300367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4302176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4304015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4305807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4307635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4309445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4311283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4313075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4314950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4316840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4318722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4320517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4322323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4324148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4325971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4327789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4329597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4331421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4333241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4335056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4336859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4338721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4340624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4342478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4344267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4346094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4347908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4349697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4351521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4353370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4355205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4356995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4359398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4361287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4363227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4365176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4367077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4368897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4370748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4372552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4374384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4376200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4378053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4379865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4381674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4383498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4385315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4387175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4389065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4390945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4392773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4394588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4396396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4398218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4400043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4401863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4403676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4405502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4407322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4409134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4410984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4412888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4414776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4416571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4418404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4420224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4422074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4423875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4425702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4427504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4429338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4431132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4432951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4434797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4436716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4438554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4440373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4442180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4444011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4445820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4447622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4449454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4451278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4453131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4454942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4456781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4459053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4461018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4462962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 69%] 2024-08-08T21:00:13.4464867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4466692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4468518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4470325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4472128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4473961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4475746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4477560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4479370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4481214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4483001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4484860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4486747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4488628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4490473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4492345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4494258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4496097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4497894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4499712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4501519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4503340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4505158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4506969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4508840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4510754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4512611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4514414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4516235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4518051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4519855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4521649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4523479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4525292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4527103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4528902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4530723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4532590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4535184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4537066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4538878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4540722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4542508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4544319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4546127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4547963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4549754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4551570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4553422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4555262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4557095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4559437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4561337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4563148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4564955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4566748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4568575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4570389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4572201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4573999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4575813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4577633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4579434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4581224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4583143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4585034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4586895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4588700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4590548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4592378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4594177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4596015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4597828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4599674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4601470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4603295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4605096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4606969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4608839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4610698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4612502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4614342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4616133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4617951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4619758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4621598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4623391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4625192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4627043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4628890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4630767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4632687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4634586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4636433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4638275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4640114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4641966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4643813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4645646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4647468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4649302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4651132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4652996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4654857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4656776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4659060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4660911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4662758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4664578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4666429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4668244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4670081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4671899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4673752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4675566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4677405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4679312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4681305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4683190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4685042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4686906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4688723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4690553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4692378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4694203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4696022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4697836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4699653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4701500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4703375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4705272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4707133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4708970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4710798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4712628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4714447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4716287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4718117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4719917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4721750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4723580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4725434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4727285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4729228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4731097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4732947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4734756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4736587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4738391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4740232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4742025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4743857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4745673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4747522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4749321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4751179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4753147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4755029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4756850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4759096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4760985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4762822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4764649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4766473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4768318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4770158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4771991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4773816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4775755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4777726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4779632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4781461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4783284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4785129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4786924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4788754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4790594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4792448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4794254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4796089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4797910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4799807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4801700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4803583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4805411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4807275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4809083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4810924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4812760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4814613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4816456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4818286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4820145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4821994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4823887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4825800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4827675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4829501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4831326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4833146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4834988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4836823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4838651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4840470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4842312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4844150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4845965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4847853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4849761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4851654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4853497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4855342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4857168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4859411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4861249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4863094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4864917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4866773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4868586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4870425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4872318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4874276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4876133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4877942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4879782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4881619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4883442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4885255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4887096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4888917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4890733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4892549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4894389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4896264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4898184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4900073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4901917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4903756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4905599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4907429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4909262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4911124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4912937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4914782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4916597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4918441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4920283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4922196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4924065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4925909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4927710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4929548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4931371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4933216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4935023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4936864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4938680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4940509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4942333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4944230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4946190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4948088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4949902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4951723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4953615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4955460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4957292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4959511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4961376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4963195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4965007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4966819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4968656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4970577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4972504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4974395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4976217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4978067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4979874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4981708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4983538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4985382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4987184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4989019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4990848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4992701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4994556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4996502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.4998377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5000234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5002078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5003902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5005730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5007550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5009372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5011181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5013017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5014849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5016668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5018526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5020443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5022309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5024132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5025945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5027784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5029611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5031438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5033253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5035083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5036940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5038758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5040593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5042459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5044403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5046260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5048100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5049906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5051740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5053574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5055404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5057220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5059490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5061326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5063161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5064978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5066897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5068844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5070733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5072576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5074407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5076225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5078057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5079882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5081733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5083534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5085354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5087194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5089026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5090889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5092808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5094675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5096494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5098314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5100112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5101923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5115803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5117716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5119546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5121362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5123206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5124995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5126919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5128842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5130740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5132540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5134372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5136194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5138039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5139842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5141661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5143497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5145325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5147149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5148970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5150835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5152808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5154663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5156470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5158308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5160645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5162467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5164271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5166103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5167917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5169726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5171535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5173348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5175301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5177231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5179124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5180938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5182786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5184592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5186423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5188242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5190083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5191880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5193707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5195506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5197333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5199177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5201094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5202959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5204779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5206596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5208411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5210235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5212052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5213867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5215670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5217496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5219313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5221120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5222963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5224881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5226756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5228576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5230395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5232231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5234058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5235858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5237689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5239492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5241323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5243112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5244933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5246783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5248709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5250545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5252407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5254223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5256061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5257849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5260075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5261920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5263759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5265548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5267349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5269173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5270985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5272888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5274835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5276748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5278574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5280388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5282189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5284003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5285816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5287615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5289405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5291227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5293043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5294826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5296690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5298582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5300456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5302248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5304070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5305882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5307720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5309524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5311347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5313161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5315006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 70%] 2024-08-08T21:00:13.5316811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5318637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5320486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5322419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5324274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5326088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5327912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5329722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5331529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5333333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5335152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5336952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5338738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5340550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5342375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5344231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5346131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5347983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5349815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5351634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5353498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5355307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5357135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5359409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5361242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5363080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5364898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5366747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5368631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5370607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5372473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5374296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5376083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5377901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5379716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5381551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5383344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5385163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5386967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5388786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5390597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5392449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5394361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5396225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5398031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5399829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5401655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5403470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5405287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5407094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5408920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5410737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5412553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5414352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5416210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5418104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5419936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5421742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5423548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5425377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5427159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5428971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5430773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5432589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5434371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5436187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5437995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5439872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5441759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5443627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5445438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5447280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5449086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5450902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5452781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5454621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5456443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5458251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5460445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5462264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5464157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5466099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5467999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5469816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5471629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5473438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5475266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5477080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5478899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5480701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5482532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5484347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5486180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5488010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5489917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5491801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5493603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5495432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5497302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5499085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5500879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5502702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5504499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5506329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5508119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5509930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5511725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5513584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5515487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5517332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5519156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5520976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5522783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5524583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5526410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5528225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5530030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5531833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5533659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5535472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5537321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5539217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5541092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5542913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5544720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5546546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5548342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5550166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5551950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5553796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5555610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5557449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5559602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5561546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5563473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5565366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5567156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5568970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5570781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5572595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5574406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5576204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5578029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5579852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5581668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5583479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5585355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5587261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5589121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5590922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5592737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5594539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5596336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5598128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5599947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5601760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5603546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5605357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5607154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5609030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5610905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5612766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5614571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5616399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5618181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5619983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5621789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5623622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5625404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5627223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5629029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5630863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5632702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5634598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5636469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5638269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5640061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5641843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5643656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5645463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5647263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5649053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5650869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5652721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5654527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5656360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5658274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5660575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5662392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5664195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5666011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5667848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5669640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5671466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5673278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5675116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5676914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5678732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5680602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5682560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5684419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5686225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5688027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5689869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5691658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5693452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5695277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5697091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5698898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5700700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5702528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5704344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5706198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5708084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5709964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5711792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5713608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5715417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5717247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5719064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5720882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5722685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5724490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5726318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5728101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5729954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5731862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5733741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5735533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5737350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5739154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5740981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5742775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5744601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5746405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5748240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5750038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5751853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5753748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5755663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5758180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5760387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5762233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5764058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5765872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5767678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5769490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5771288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5773083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5774874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5776693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5778591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5780520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5782385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5784208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5786023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5787810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5789628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5791428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5793254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5795036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5796845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5798646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5800480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5802316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5804219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5806072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5807909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5809704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5811517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5813310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5815130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5816912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5818702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5820518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5822328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5824127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5825962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5827882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5829737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5831531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5833320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5835144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5836956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5838769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5840573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5842404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5844225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5846022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5847842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5849694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5851627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5853514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5855331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5857126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5859311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5861127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5862935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5864745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5866572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5868355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5870180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5871973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5873791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5875658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5877596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5879481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5881299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5883114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5884911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5886735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5888563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5890385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5892198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5894028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5895849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5897655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5899502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5901403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5903260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5905062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5906850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5908659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5910495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5912290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5914108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5915909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5917740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5919529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5921335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5923196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5925113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5926939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5928756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5930566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5932402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5934191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5936015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5937825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5939637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5941452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5943259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5945073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5946918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5948795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5950629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5952482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5954306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5956105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5957909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5960134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5961980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5963779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5965573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5967388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5969223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5971098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5973052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5974929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5976764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5978560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5980385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5982190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5984022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5985821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5987637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5989429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5991247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5993030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5994878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5996767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.5998641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6000439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6002232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6004058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6005870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6007672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6009469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6011285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6013087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6014887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6016676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6018537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6020434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6022280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6024073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6025890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6027706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6029499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6031315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6033113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6034933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6036707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6038512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6040309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6042129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6043948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6045840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6047681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6049507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6051288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6053134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6054935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6056765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6058887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6060724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6062549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6064356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6066151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6068033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6069984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6071939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6073680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6075480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6077292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6079095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6080896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6082761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6084489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6086304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6088090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6089897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6091735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6093671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6095505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6097312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6099111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6100943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6102727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6104539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6106341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6108180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6109964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6111784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6113590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6115463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6117340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6119185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6121002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6122807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6124602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6126387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6128206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6130022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6131830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6133625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6135451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6137252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6139098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6140981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6142849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6144662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6146468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 71%] 2024-08-08T21:00:13.6148258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6150052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6151880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6153710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6155534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6157337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6159515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6161326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6163215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6165130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6167010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6168774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6170565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6172355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6174177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6175945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6177732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6179543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6181340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6183137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6184929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6186738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6188591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6190492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6192325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6194148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6195971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6197774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6199577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6201395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6203202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6205000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6206801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6208598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6210408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6212227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6214115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6215958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6217782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6219567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6221374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6223191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6225019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6226793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6228589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6230401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6232244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6234053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6235936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6237849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6239732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6241561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6243377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6245214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6247050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6248878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6250695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6252554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6254374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6256190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6257994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6260305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6262280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6264173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6265988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6267823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6269645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6271440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6273267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6275079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6276916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6278714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6280546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6282366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6284277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6286171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6288050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6289863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6291713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6293518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6295346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6297153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6298985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6300773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6302577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6304414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6306235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6308091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6309997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6311874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6313698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6317291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6320811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6324334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6327919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6331406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6334938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6338563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6342041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6345563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6349124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6352791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6356435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6360230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6363787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6367300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6370823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6374332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6377830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6381343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6384857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6388369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6391861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6395464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6399128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6402730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6406266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6409837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6413384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6416951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6420476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6424024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6427584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6431112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6434645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6438179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6441790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6445427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6449014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6452598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6456126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6459983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6463517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6467051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6470602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6474141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6477723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6481165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6484712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6488416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6491961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6495557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6499184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6502631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6506161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6509684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6513243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6516785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6520320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6523859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6527408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6530957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6534531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6538150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6541723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6545233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6548734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6552242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6555847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6559771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6563321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6566858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6570409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6573917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6577430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6581045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6584736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6588324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6591844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6595378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6598913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6602447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6606108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6610147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6613734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6617270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6620789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6624293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6627901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6631508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6635058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6638570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6642112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6645612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6649116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6652691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6656237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6660161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6663708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6667251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6670780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6674425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6678152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6681742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6685302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6688850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6692388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6695921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6699489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6703046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6706575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6710109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6713643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6717156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6720705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6724344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6727938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6731471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6734992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6738531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6742061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6745582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6749100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6752676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6756241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6760152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6763703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6767217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6770885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6774576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6778172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6782362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6785962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6789711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6793258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6796793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6800319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6803824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6807324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6810841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6814345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6817935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6821530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6825101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6828646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6832174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6835678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6839174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6842728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6846249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6849770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6853355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6856917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6860914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6864524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6868203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6871839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6875383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6878913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6882435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6885945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6889463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6892951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6896465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6900007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6903526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6907027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6910578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6914216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6917795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6921362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6924895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6928439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6931966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6935464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6939007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6942557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6946089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6949620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6953199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6956818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6960865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6964467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6967984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6971511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6975027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6978518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6982013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6985550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6989064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6992576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6996112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.6999650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7003233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7006823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7010392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7013920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7017498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7020942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7024462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7028011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7031507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7035006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7038533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7042072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7045596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7049157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7052822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7056397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7060191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7063711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7067221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7070752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7074272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7077750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7081259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7084777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7088384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7091795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7095397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7099170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7102669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7106186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7109728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7113282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7116809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7120336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7123859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7127411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7130948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7134459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7137983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7141562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7145155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7148679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7152198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7155777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7159671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7163212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7166732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7170274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7173782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7177295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7180954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7184418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7188024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7191671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7195246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7198803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7202325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7205850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7209390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7212947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7216479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7219983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7223504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7227029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7230537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7234089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7237693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7241279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7244783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7248295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7251824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7255411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7259269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7262814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7266327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7269872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7273404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7276910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7280530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7284242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7287841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7291339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7294875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7298418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7301945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7305455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7308973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7312501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7315995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7319502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7323017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7326552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7330110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7333703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7337254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7340784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7344313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7347812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7351333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7354925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7358795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7362311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7365852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7369395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7372913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7376505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7380165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7383780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7387295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7390820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7394344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7397868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7401374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7404863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7408345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7411869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7415391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7418899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7422453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7426086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7429643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7433121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7436640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7440179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7443708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7447223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7450751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7454338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7457855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7461847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7465385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7469028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7472707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7476290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7479811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7483330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7486841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7490336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7493863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7497400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7500912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7504395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7507919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7511456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7515021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7518610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7522173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7525702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7529197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7532711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7536245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7539814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7543349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7546854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7550382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7554389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7557968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7562017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7565682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7569281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7573097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7576761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7580293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7583817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7587332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7590844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7594366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7597898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7601406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7604915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7608496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 72%] 2024-08-08T21:00:13.7613124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7616694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7620196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7623728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7627254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7630784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7634287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7637815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7641360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7644883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7648387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7651905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7655533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7659615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7663215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7666730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7670266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7673769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7677278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7680790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7684326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7687841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7691342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7694866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7698405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7701995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7705616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7709194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7712755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7716289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7719798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7723334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7726876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7730414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7733933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7737450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7740992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7744487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7748038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7751644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7755281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7759193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7762747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7766285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7769805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7773321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7776826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7780346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7783880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7787395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7790890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7794411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7798040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7801689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7805286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7808807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7812356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7815865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7819380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7822891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7826407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7829905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7833399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7836901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7840413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7843975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7847569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7851133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7854702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7858215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7862103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7865627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7869146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7872658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7876159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7879682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7883221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7886727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7890328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7893977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7897588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7901111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7904627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7908150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7911661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7915165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7918651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7922151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7925675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7929188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7932676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7936241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7939852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7943413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7946900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7950419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7953987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7957488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7961403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7964928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7968468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7971987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7975505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7979006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7982624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7986294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7989873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7993388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.7996920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8000409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8003874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8007373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8010910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8014418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8017919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8021426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8024975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8028491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8032076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8035624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8039149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8042665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8046152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8049639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8053220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8056744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8060625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8064155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8067683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8071194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8074754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8078405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8081991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8085481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8088971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8092472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8095989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8099478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8101293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8103090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8104918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8106699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8108508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8110367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8112293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8114130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8115944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8117758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8119611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8121407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8123213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8125041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8126862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8128671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8130485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8132300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8134103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8135940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8137846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8139714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8141528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8143339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8145138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8146956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8148777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8150590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8152387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8154259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8156117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8157912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8160267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8162246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8164160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8165968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8167803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8169627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8171474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8173282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8175116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8176916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8178749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8180537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8182356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8184211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8186128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8187987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8189801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8191633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8193451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8195273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8197077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8198908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8200732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8202549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8204362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8206190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8208063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8209973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8211833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8213671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8215506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8217310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8219144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8220954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8222786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8224576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8226391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8228200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8230040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8231896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8233807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8235674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8237511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8239302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8241121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8242924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8244768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8246553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8248352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8250178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8252003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8253861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8255718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8257644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8259937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8261789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8263603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8265420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8267226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8269035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8270832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8272646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8274451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8276264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8278063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8279975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8281944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8283806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8285625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8287443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8289282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8291086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8292918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8294741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8296592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8298391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8300225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8302038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8303930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8305826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8307790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8309605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8311418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8313226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8315023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8316854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8318673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8320484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8322298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8324131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8325949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8327806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8329702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8331576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8333398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8335221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8337030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8338865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8340696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8342497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8344330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8346217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8348007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8349887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8351694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8353630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8355510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8357387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8359499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8361344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8363182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8364975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8366807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8368616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8370454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8372252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8374054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8375969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8377904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8379778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8381587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8383423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8385245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8387053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8388859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8390686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8392508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8394321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8396131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8397948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8399793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8401695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8403534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8405336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8407167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8408952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8410771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8412570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8414403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8416198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8418012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8419828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8421669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8423516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8425431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8427293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8429148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8430953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8432786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8434597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8436449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8438254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8440068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8441886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8443701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8445507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8447340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8449253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8451118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8452923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8454773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8456609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8458670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8460583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8462397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8464223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8466036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8467850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8469660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8471559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8473530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8475392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8477211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8479022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8480874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8482669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8484494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8486301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8488128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8489965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8491740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8493545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8495423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8497323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8499172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8501001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8502817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8504639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8506438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8508262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8510077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8511889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8513693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8515531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8517352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8519205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8521101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8522971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8524795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8526612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8528424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8530224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8532053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8533840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8535652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8537457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8539293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8541086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8542899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8544748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8546662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8559844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8561842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8563705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8565538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8567335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8569165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8570987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8572829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8574637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8576463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8578276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8580267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8582207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8584107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8585910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8587746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8589529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8591341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8593228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8594991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8596780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8598584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8600415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8602225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8604090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8605981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8607849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8609660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8611471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8613273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8615097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8616916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8618727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8620523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8622342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8624163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8625978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8627821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8629701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8631576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8633357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8635167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8636964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8638797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8640583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8642394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8644189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8646015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8647798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8649603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8651458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8653438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8655293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8657118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8659394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8661262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8663079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8664897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8666724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8668543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8670360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8672177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8673995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8675897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8677834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8679723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8681547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8683368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8685178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8686976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8688804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8690623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 73%] 2024-08-08T21:00:13.8692414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8694233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8696047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8697885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8699721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8701675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8703547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8705386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8707187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8709014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8710821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8712659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8714464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8716293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8718089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8719922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8721707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8723544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8725453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8727320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8729133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8730937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8732759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8734574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8736377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8738179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8740008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8741830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8743647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8745447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8747324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8749236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8751105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8752916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8754775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8756628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8758790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8760669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8762473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8764304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8766089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8767903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8769707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8771624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8773555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8775440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8777248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8779089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8780890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8782686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8784507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8786321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8788125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8789922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8791755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8793579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8795432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8797321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8799191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8801003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8802810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8804615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8806428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8808225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8810018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8811809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8813608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8815436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8817226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8819038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8820879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8822786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8824614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8826422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8828234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8830066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8831855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8833676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8835488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8837328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8839129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8840955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8842763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8844628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8846525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8848383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8850199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8852008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8853847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8855651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8857473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8859647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8861482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8863289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8865114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8866921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8868810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8870753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8872643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8874456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8876255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8878077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8879892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8881745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8883541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8885360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8887172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8889011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8890814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8892686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8894572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8896440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8898225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8900040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8901844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8903679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8905477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8907280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8909097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8910910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8912721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8914524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8916408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8918322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8920181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8921979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8923811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8925628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8927437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8929244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8931072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8935848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8940049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8941894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8943700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8945628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8947518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8949336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8951167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8952995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8954848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8956655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8958876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8960790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8962606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8964572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8966477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8968308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8970160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8972054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8973862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8975707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8977505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8979323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8981130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8982976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8984776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8986597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8988464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8990350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8992132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8993984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8995836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8997644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.8999459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9001271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9003092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9004906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9006719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9008524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9010335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9012199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9014067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9015868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9017744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9019618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9021427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9023224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9025060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9026879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9028665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9030488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9032300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9034119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9035944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9037802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9039601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9041474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9043323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9045139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9046939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9048770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9050552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9052364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9054215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9056057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9057840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9060136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9062077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9063897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9065722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9067596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9070162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9071981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9073797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9075610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9077419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9079233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9081038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9082826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9084689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9086615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9088415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9090213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9092066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9093942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9095730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9097545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9099354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9101179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9102972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9104795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9106607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9108482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9110330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9112147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9113950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9115832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9117686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9119504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9121308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9123120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9124911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9126704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9128529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9130338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9132182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9134038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9135861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9137672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9139535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9141381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9143195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9145007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9146823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9148611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9150412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9152248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9154079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9155906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9157760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9160009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9161817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9163726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9165603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9167415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9169191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9171000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9172788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9174610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9176401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9178209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9179998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9181879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9183746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9185531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9187392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9189259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9191061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9192867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9194704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9196522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9198330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9200138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9201970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9203784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9205645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9207500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9209315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9211164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9212993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9214794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9216593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9218435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9220221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9222036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9223838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9225673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9227454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9229312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9231186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9233010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9234833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9236696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9238493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9240321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9242107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9243913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9245725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9247536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9249341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9251132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9252984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9254886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9256679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9258801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9260795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9262678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9264472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9266256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9268079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9269887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9271678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9273497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9275284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9277184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9279043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9280850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9282652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9284535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9286373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9288183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9289987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9291825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9293610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9295430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9297231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9299064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9300857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9302697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9304508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9306307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9308153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9309993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9311798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9313605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9315407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9317192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9319005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9320819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9322615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9324390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9326262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9328132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9329929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9331765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9333622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9335442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9337231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9339040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9340823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9342633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9344411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9346210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9347996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9349879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9351719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9353557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9355403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9357279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9359422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9361235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9363061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9364877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9366682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9368469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9370294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9372109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9373996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9375869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9377685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9379497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9381358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9383216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9385027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9386835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9388638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9390419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9392214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9394040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9395812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9397659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9399505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9401324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9403097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9404958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9406821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9408642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9410436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9412250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9414047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9415881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9417671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9419469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9421326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9423191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9424999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9426791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9428640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9430492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9432275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9434065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9435889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9437690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9439488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9441282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9443088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9444888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9446724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9448566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9450363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9452228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9454097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9455905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9457704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9459918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9461728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9463534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9465337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9467167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9468942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9470889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9472753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9474570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9476402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9478264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9480049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9481853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9483654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9485437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9487249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9489156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9490862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9492637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9494494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9496355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9498153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9500006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9501881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9503693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9505488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9507292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9509104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9510928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9512721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9514544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9516337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9518198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9520025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9521824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9523617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9525498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9527322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 74%] 2024-08-08T21:00:13.9529115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9530916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9532750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9534521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9536335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9538149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9539955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9541797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9543635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9545448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9547253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9549097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9550949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9552760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9554617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9556430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9558217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9560723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9562554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9564347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9566222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9568105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9569909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9571677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9573540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9575407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9577227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9579002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9580810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9582594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9584415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9586199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9587999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9589800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9591678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9593526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9595335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9597167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9599028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9600822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9602682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9604419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9606207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9607985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9609762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9611573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9613367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9615204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9617038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9618845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9620683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9622516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9624298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9626103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9627919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9629696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9631493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9633288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9635119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9636892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9638737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9640588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9642410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9644191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9646048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9647880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9649686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9651457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9653265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9655098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9656909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9659092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9660918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9662808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9664685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9666482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9668268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9670133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9671999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9673786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9675564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9677387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9679190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9680983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9682775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9684580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9686373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9688216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9690071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9691858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9693710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9695528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9697320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9699105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9701000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9702692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9704491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9706282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9708097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9709862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9711704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9713548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9715352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9717193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9719028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9720845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9722659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9724469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9726265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9728075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9729899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9731715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9733503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9735363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9737219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9739008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9740781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9742640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9744489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9746262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9748066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9749864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9751669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9753446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9755304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9757115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9759422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9761329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9763154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9764969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9766876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9768764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9770591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9772403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9774251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9776047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9777860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9779686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9781497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9783344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9785190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9787008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9788817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9790670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9792527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9794344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9796161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9797978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9799771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9801597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9803422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9805228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9807079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9808949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9810792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9812583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9814455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9816315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9818151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9819953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9821782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9823576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9825401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9827190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9829000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9830844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9832730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9834521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9836316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9838182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9840098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9841901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9843703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9845532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9847338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9849150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9850956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9852777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9854635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9856505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9858710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9860629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9862554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9864451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9866252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9868056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9869895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9871672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9873487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9875297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9877128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9878910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9880795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9882674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9884505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9886336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9888200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9890006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9891840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9893637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9895465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9897278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9899110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9900928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9902736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9904615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9906488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9908305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9910158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9912029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9913838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9915641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9917439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9919259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9921073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9922883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9924692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9926523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9928381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9930220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9932022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9933877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9935767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9937556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9939375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9941199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9943045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9944837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9946673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9948496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9950338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9952183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9954112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9955908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9957783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9960034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9961856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9963688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9965507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9967318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9969121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9970954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9972781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9974592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9976470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9978363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9980175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9982036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9983902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9985725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9987541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9989356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9991165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9992963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9994800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9996599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:13.9998416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0000255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0002123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0003898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0005708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0007547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0009421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0011197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0013017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0014825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0016649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0018439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0020261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0022067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0023929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0025795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0027594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0029420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0031290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0033149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0034958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0036791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0038623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0040431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0042243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0044066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0045867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0047729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0049573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0051400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0053215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0055091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0056967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0059141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0061019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0062822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0064636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0066439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0068272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0070060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0071962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0073851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0075694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0077491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0079374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0081293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0083093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0084891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0086707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0088520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0090330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0092191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0093930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0095785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0097646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0099462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0101261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0103121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0104987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0106787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0108576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0110406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0112219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0114031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0115838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0117651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0119485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0121334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0123213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0125020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0126897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0128755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0130572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0132372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0134202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0135980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0137791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0139600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0141436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0143219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0145064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0146932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0148746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0150599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0152450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0154305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0156122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0157935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0160184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0162041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0163871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0165690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0167490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0169403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0171300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0173110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0174971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0176845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0178666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0180443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0182254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0184060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0185888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0187672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0189500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0191296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0193170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0195006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0196810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0198647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0200529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0202309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0204115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0205920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0207744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0209542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0211345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0213172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0214979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0216823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0218669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0220475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0222271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0224163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0225995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0227799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0229607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0231417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0233199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0235016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0236823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0238596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0240443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0242291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0244124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0245903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0247765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0249622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0251455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0253252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0255120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0256929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0259148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0260984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0262807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0264700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0266596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0268382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0270169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0272043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0273924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0275720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0277527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0279357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0281165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0282972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0284771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0286587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0288390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0290247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0292742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0294575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0296443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0298293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0300113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0301925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0303772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0305566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0307387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0309194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0311017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0312795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0314654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0316529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0318359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0320191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0322059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0323858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0325694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0327492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0329287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0331103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0332919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0334737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0336528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0338402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0340277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0342088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0343932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0345810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0347621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0349434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0351247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0353066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0354912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0356720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0358856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0360697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 75%] 2024-08-08T21:00:14.0362620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0364493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0366306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0368168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0370064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0371848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0373652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0375454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0377283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0379063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0380877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0382689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0384517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0386352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0388215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0390016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0391823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0393678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0395519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0397325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0399128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0400928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0402710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0404523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0406342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0408135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0409979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0411851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0413654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0415448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0417282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0419136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0420958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0422748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0424580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0426388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0428231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0430028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0431847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0433690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0435576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0437368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0439181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0441016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0442894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0444676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0446489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0448299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0450106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0451907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0453751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0455569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0457428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0459679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0461509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0463326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0465236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0467136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0468938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0470776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0472607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0474421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0476223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0478065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0479887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0481678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0483567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0485435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0487253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0489082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0490956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0492757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0494586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0496383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0498197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0500005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0501838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0503629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0505445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0507312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0509226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0511010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0512855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0514730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0516540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0518346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0520151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0521975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0523785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0525594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0527402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0529213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0531058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0532903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0534690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0536546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0538412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0540195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0541997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0543798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0545626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0547399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0549210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0551020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0552853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0554727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0556606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0558790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0560670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0562557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0564451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0566259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0568099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0569907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0571715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0573534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0575349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0577146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0579002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0580885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0582691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0584500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0586359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0588227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0590034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0591842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0593642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0595451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0597262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0599068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0600851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0602692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0604629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0606416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0608227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0610072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0611953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0613739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0615556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0617356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0619168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0620947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0622758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0624547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0626409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0628242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0630046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0631838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0633681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0635533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0637323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0639138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0640953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0642747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0644531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0646355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0648173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0649971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0651816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0653682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0655526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0657379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0659634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0661462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0663284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0665070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0666874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0668671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0670500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0672277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0674074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0675954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0677872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0679651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0681514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0683382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0685203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0686984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0688796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0690600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0692408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0694218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0696020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0697838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0699699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0701556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0703348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0705203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0707050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0708840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0710619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0712441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0714256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0716054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0717840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0719661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0721469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0723289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0725155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0726948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0728777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0730602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0732449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0734244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0736074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0737866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0739669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0741469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0743296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0745065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0746907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0748743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0750541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0752328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0754191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0756053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0757855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0760035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0761853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0763669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0765470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0767266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0769048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0770859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0772759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0774640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0776436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0778317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0780202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0781988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0783811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0785622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0787453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0789250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0791075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0792873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0794687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0796511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0798373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0800161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0802036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0803869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0805679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0807477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0809315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0811093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0812883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0814711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0816535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0818333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0820168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0822049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0823868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0825714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0827575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0829394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0831205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0833021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0834822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0836633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0838433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0840221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0842021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0843870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0845743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0847522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0849326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0851167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0853052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0854867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0856685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0858843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0860709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0862494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0864308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0866110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0868026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0869890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0871681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0873497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0875368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0877232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0879030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0880836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0882639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0884424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0886211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0888031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0889836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0891689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0893531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0895344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0897141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0898982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0900823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0902613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0904443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0906228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0908026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0909824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0911654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0913427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0915229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0917071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0918950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0920726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0922574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0924411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0926216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0927985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0929778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0931582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0933384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0935184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0936968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0938769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0940607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0942459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0944231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0946097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0947955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0949809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0951595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0953418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0955278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0957084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0959240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0961082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0962908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0964778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0966676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0968479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0970350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0972187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0973984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0975775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0977610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0004s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0979386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0981191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0982996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0984831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0986608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0988525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0990377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0992186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0993992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0995822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0997692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.0999508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1001313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1003117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1004934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1006751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1008562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1010356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1012213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1014061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1015851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1017627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1019468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1021352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1023136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1024942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1026751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1028568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1030349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1032163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1033959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1035828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1037678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1039484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1041280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1043150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1044981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1046793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1048589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1050406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1052198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1054025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1055839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1057640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1059820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1061710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1063594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1065391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1067242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1069092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1070901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1072703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1074504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1076284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1078096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1079912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1081698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1083500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1085346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1087232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1089020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1090878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1092728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1094553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1096340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1098156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1099950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1101763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1103540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1105352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1107144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1108997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1110845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1112629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1114479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1116333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1118123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1119904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1121719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1123532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1125322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1127105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1128927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1130733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1132592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1134442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1136253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1138062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1139889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1141745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1143527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1145341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1147119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1148907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1150698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1152520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1154328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1156171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1158016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1161524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1163413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1165334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1167210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1169014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1170810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1172615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1174434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1176253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1178056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1179843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1181665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1183535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1185405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1187201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1189048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1190892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 76%] 2024-08-08T21:00:14.1192675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1194452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1196275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1198078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1199857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1201659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1203455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1205265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1207086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1208940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1210735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1212604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1214451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1216249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1218041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1219871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1221651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1223454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1225257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1227093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1228876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1230709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1232571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1234364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1236191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1238018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1239820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1241615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1243410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1245195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1247003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1248805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1250606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1252382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1254275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1256138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1257908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1260110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1262030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1263921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1265689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1267495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1269288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1271105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1272883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1274686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1276460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1278334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1280169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1281963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1283741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1285574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1287411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1289186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1290988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1292790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1294574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1296350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1298164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1299968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1301790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1303583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1305452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1307255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1309102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1310945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1312756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1314568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1316361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1318173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1321708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1325207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1328704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1332148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1335758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1339310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1342768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1346259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1349771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1353238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1356735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1360647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1364145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1367657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1371146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1374629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1378103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1381713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1385290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1388781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1392353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1395987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1399479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1402943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1406439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1409944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1413424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1416897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1420393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1423892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1427412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1430943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1434433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1437932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1441470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1444979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1448450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1451953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1455494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1459376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1462924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1466461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1469946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1473549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1477097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1480608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1484105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1487654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1491199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1494689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1498158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1501630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1505108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1508608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1512091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1515556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1519089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1522621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1526106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1529579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1533114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1536664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1540142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1543596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1547100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1550599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1554132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1557685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1561533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1565120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1568678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1572172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1575654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1579213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1582751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1586210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1589693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1593175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1596650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1600123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1603612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1607098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1610621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1614113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1617597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1621094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1624639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1628180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1631672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1635188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1638684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1642167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1645658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1649176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1652683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1656215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1660222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1663814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1667294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1670824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1674969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1678478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1681963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1685414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1688895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1692394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1695885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1699349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1702830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1706381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1709897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1713374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1716915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1720485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1723980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1727468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1730977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1734471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1737972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1741461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1744938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1748429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1751957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1755501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1759381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1763010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1766570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1770036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1773530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1777042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1780511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1783976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1787467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1790971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1794451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1797994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1801559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1805053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1808596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1812136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1815630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1819159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1822785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1826590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1830071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1833556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1837004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1840450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1843982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1847531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1851007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1854565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1858086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1862009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1865489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1868951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1872439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1875943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1879426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1882891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1886378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1889981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1893560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1897041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1900543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1904125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1907677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1911157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1914638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1918311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1921878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1925346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1928827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1932404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1935921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1939494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1942982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1946490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1950028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1953531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1957048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1961203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1964630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1968099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1981789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1985362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1988852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1992494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1996065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.1999572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2003054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2006621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2010185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2013659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2017125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2020559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2024020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2027517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2030996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2034450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2038013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2041526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2044983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2048468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2051957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2055561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2059519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2063002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2066477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2069981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2073470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2076947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2080436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2084034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2087601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2091058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2094536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2098086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2101613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2105060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2108526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2112020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2115478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2118942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2122421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2125913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2129382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2132893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2136396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2139891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2143462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2146979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2150476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2153977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2157524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2161453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2164901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2168405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2171897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2175368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2178943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2182533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2185972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2189495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2193036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2196528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2200006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2203482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2206939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2210429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2213903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2217374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2220877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2224397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2227916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2231358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2234880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2238426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2241897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2245352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2248838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2252331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2255836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2259779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2263294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2266763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2270311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2273835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2277273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2280750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2284309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2288227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2291642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2295125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2298595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2302037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2305522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2309022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2312504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2316033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2319579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2323070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2326548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2330064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2333603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2337086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2340578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2344065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2347526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2351006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2354515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2357971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2361981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2365558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2369032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2372481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2376024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2379587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2383065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2386737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_1024_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2390289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2393810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2397343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2400814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2404332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2407917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2411498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2415072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2418535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2422167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2425656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2429227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2432660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2436185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2439668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2443145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2446635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2450143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2453638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2457242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2461252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2464779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2468352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2471945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2475470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2478953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2482548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2485954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2489470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2492993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2496508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2500001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2503580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2507186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2510685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2514237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2517798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2521302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2524788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2528273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2531763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2535271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2538758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2542258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2545753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2549315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2552873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2556401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2560413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2564020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2567521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2571007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2574516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2578041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2581541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2585035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2588533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2592053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2595632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2599201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2602696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2606251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2609763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2613239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2616815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2620259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2623790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2627266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2630760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2634261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2637744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2641290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2644845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2648359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2651871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2655496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2659562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2663140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2666665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2670184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2673690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2677222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2680717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2684218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 77%] 2024-08-08T21:00:14.2687816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2691407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2694904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2698382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2701960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2705533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2709036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2712533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2716046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2719558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2723064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2726559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2730042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2733652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2737223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2740720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2744224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2747801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2751351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2754893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2758884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2762427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2765936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2769445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2772957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2776440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2780033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2783606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2787105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2790624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2794202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2797770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2801263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2804790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2808288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2811781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2815290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2818806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2822283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2825765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2829320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2832916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2836426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2839976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2843535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2847039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2850545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2854046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2857594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2861615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2865109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2868564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2872048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2875655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2879332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2882810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2886381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2889967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2893435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2896913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2900413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2903937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2907434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2910934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2914467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2917987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2921562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2925115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2928624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2932207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2935772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2939261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2942771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2946280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2949774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2953250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2956798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2960769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2964259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2967853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2971447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2974967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2978472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2982029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2985588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2989092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2992596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2996081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.2999585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3003111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3006627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3010098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3013673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3017243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3020746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3024241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3027786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3031334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3034788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3038272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3041772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3045278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3048779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3052255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3055794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3059855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3063470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3066961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3070456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3074034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3077594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3081057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3084566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3088103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3091606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3095102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3098613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3100428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3102292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3104158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3105977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3107782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3109620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3111466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3113263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3115099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3116883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3118687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3120485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3122313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3124096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3125897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3127747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3129630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3131413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3133267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3135126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3136959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3138755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3140565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3142410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3144233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3146043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3147844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3149656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3151505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3153361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3155192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3157060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3159329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3161207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3163000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3164823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3166634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3168427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3170220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3172019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3173836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3175707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3177585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3179385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3181277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3183137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3184944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3186734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3188569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3190352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3192150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3193943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3195771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3197541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3199385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3201231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3203034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3204833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3206666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3208531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3210328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3212124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3213917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3215732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3217549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3219358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3221146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3223016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3224890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3226703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3228503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3230349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3232233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3234024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3235854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3237658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3239474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3241252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3243067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3244854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3246694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3248521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3250385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3252182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3254068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3255947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3257762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3259945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3261805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3263620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3265433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3267279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3269107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3270927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3272830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3274748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3276573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3278452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3280335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3282153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3283967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3285784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3287582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3289409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3291239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3293036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3294858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3296719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3298610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3300403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3302271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3304136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3305969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3307769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3309602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3311419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3313262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3315064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3316902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3318714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3320603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3322459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3324274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3326134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3328001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3329801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3331600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3333438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3335266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3337078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3338893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3340732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3342553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3344412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3346288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3348119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3349979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3351843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3353642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3355495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3357348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3359518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3361378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3363195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3365041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3366832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3368788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3370677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3372500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3374345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3376224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3378023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3379850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3381643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3383469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3385270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3387088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3388895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3390685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3392563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3394440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3396252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3398055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3399934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3401806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3403621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3405433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3407265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3409088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3410911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3412731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3414548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3416401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3418241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3420050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3421857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3423733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3425579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3427402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3429215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3431058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3432846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3434670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3436483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3438321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3440171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3442051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3443872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3445719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3447561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3449425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3451258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3453086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3454949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3456762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3458980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3460882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3462695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3464584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3466499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3468330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3470144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3472010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3473909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3475730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3477531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3479358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3481163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3483000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3484797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3486611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3488475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3490367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3492170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3493994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3495847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3497734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3499534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3501365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3503175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3504990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3506764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3508564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3510386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3512202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3514059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3515925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3517749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3519609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3521568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3523435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3525310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3527142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3528958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3530764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3532607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3534437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3536254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3538119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3539999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3541841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3543687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3545573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3547372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3549210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3551004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3552820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3554676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3556528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3558327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3560578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3562539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3564398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3566191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3568079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3569952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3571764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3573639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3575389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3577224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3579047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3580860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3582675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3584507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3586375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3589043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3590847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3592715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3594611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3596370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3598161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3599985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3601802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3603596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3605505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3607227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3609052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3610888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3612752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3614554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3616445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3618289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3620110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3621914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3623757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3625557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3627376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3629184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3631027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3632822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3634675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3636542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3638350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3640148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3641984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3643851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3645661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3647471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3649279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3651098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3652917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3654762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3656565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3658781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3660751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3662557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3664358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3666236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3668142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3669947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3671778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3673598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3675435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3677234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3679061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3680856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3682733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3684565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3686381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3688189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3690059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3691898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3693705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3695537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3697363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3699171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3700973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3702797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3704610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3706455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3708315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3710141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3711956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3713805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3715657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3717482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 78%] 2024-08-08T21:00:14.3719309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3721123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3722920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3724713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3726535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3728315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3730123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3731966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3733859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3735645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3737498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3739405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3741229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3743013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3744831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3746639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3748477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3750274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3752097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3753910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3755820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3757697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3759917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3761858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3763766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3765571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3767373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3769194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3771002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3772795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3774589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3776412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3778225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3780098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3781970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3783788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3785646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3787490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3789297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_2048_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3791102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3792938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3794729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3796549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3798365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3800215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3802005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3803885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3805750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3807583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3809380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3811243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3813085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3814908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3816693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3818487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3820304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3822119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3823925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3825727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3827591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3829464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3831265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3833058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3834924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3836788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3838590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3840392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3842233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3844051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3845852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3847683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3849488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3851369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3853216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3855076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3856875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3859188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3861084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3862893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3864705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3866546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3868333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3870158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3871971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3873812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3875667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3877539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3879364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3881175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3883033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3884897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3886724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3888544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3890354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3892154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3893980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3895806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3897609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3899409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3901272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3903134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3904928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3906765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3908616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3910444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3912239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3914050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3915843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3917670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3919455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3921263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3923069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3924955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3926798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3928620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3930485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3932383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3934176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3936008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3937834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3939652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3941462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3943277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3945105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3946913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3948755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3950609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3952433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3954293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3956190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3957991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3960192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3962041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3963854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3965647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3967451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3969290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3971088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3972996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3974897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3976907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3978784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3980682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3982489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3984331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3986132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3987963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3989765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3991597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3993388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3995198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3997053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.3998921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4000728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4002529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4004394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4006261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4008069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4009877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4011708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4013518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4015331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4017133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4018955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4020823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4022688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4024488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4026311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4028182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4030021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4031844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4033652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4035479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4037250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4039059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4040864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4042695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4044524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4046387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4048188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4050012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4051839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4053708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4055556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4057405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4059623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4061463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4063302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4065132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4066939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4068740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4070659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4072572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4074381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4076248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4078138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4079938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4081741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4083536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4085351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4087166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4088976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4090785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4092586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4094466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4096308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4098119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4099961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4101839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4103626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4105439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4107248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4109080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4110873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4112690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4114488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4116317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4118149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4120002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4121811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4123708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4125563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4127346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4129163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4130974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4132778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4134558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4136374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4138189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4139982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4141826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4143692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4145503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4147348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4149194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4151003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4152836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4154631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4156501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4158311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4160565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4162369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4164185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4166082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4167990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4169772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4171586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4173445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4175344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4177131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4178946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4180755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4182564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4184368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4186164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4187978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4189838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4191697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4193497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4195319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4197181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4199044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4200845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4202675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4204499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4206306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4208112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4209915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4211731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4213549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4215412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4217213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4219045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4220874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4222736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4224533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4226364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4228149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4229956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4231758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4233585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4235366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4237170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4239019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4240883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4242680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4244522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4246406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4248218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4250029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4251823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4253632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4255466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4257274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4259447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4261291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4263185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4265063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4266846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4268715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4270592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4272364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4274169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4275975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4277806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4279585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4281399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4283214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4285044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4286882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4288756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4290556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4292388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4294219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4296087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4297887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4299689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4301484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4303264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4305083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4306897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4308700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4310531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4312400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4314208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4316001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4317837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4319700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4321511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4323315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4325110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4326926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4328744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4330530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4332337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4334179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4336052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4337837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4339644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4341494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4343363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4345133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4346935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4348734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4350557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4352333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4354143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4355974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4357775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4360149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4362046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4363855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4365722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4367592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4369380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4371195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4373014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4374817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4376607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4378437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4380249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4382050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4383889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4385755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4387549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4389367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4391216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4393010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4394836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4396619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4398427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4400224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4402042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4403827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4405627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4407461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4409341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4411115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4412913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4414770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4416649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4418426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4420221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4422036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4423835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4425634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4427431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4429233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4431074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4432929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4434700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4436504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4438341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4440183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4441960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4443763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4445566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4447336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4449144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4450943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4452766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4454596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4456509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4458313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4460550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4462430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4464320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4466110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4467939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4469724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4471525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4473316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4475144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4476917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4478691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4480560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4482503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4484230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4486056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4487919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4489779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4491506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4493295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4495107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4496913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4498715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4500511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4502327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4504190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4506081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4507893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4509729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4511610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4513392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4515199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4516993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4518813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4520581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4522377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4524173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4526000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4527839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4529702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4531489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4533312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4535129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4536962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4538776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4540578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4542376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4544153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4545968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4547777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4549568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 79%] 2024-08-08T21:00:14.4551399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4553255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4555104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4556907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4559155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4561075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4562870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4564650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4566445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4568228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4570047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4571829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4573623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4575407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4577285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4579125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4580924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4582759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4584637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4586417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4588233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4590044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4591872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4593654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4595448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4597264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4599072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4600921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4602773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4604574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4606412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4608272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4610050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4611866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4613745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4615480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4617262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4619079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4620890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4622661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4624511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4626354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4628172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4629950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4631795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4633639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4635462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4637247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4639059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4640853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4642680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4644468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4646277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4648104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4649974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4651744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4653522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4655414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4657274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4659472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4661343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4663163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4664958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4666756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4668540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4670337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4672228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4674100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4675880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4677694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4679563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4681415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4683222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4685018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4686847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4688624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4690431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4692226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4694039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4695806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4697645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4699483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4701299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4703130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4704988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4706773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4708569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4710363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4712145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4713938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4715735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4717530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4719308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4721169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4723026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4724817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4726609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4728462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4730316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4732112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4733899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4735702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4737490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4739257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4741055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4742842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4744703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4746530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4748328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4750116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4751968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4753794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4755631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4757423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4759627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4761427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4763231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4765027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4766826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4768706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4770568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4772376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4774179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4776035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4777886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4779681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4781474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4783261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4785026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4786827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4788637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4790429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4792203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4794054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4795916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4797683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4799522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4801988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4803811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4805590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4807405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4809212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4811038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4812824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4814643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4816435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4818319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4820156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4821965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4823751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4825597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4827429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4829200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4831020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4832829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4834623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4836410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4838223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4840022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4841857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4843693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4845531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4847350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4849202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4851062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4852897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4854726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4856585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4858745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4860639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4862487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4864283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4866193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4868069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4869893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4871668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4873533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4875403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4877236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4879027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4880856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4882659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4884486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4886297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4888092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4889912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4891829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4893691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4895490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4897367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4899252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4901067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4902878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4904710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4906528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4908346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4910161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4911986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4913788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4915612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4917475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4919276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4921155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4923001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4924817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4926624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4928464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4930251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4932058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4933860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4935693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4937478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4939331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4941193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4943026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4944860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4946715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4948537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4950352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4952161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4953967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4955816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4957629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4959854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4961679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4963593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4965483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4967288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4969143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4971040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4972850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4974647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4976443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4978259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4980088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4981885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4983714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4985529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4987426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4989278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4991100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4992913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4994801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4996648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.4998479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5000288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5002125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5003906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5005727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5007536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5009344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5011205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5013064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5014886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5016704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5018555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5020407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5022224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5024041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5025860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5027661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5029496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5031327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5033146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5034996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5036859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5038704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5040497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5042369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5044220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5046040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5047828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5049646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5051452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5053284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5055114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5056933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5059203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5061146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5062941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5064755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5066633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5068519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5070327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5072123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5073952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5075768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5077582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5079389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5081216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5083026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5084881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5086743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5088559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5090407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5092259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5094046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5095863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5097688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5099466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5101277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5103079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5104908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5106690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5108547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5110411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5112239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5114072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5115949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5117767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5119611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5121418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5123246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5125058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5126903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5128706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5130508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5132371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5134233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5136027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5137866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5139744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5141557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5143364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5145177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5147001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5148811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5150627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5152428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5154244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5156145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5158009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5160179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5162019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5163945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5165822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5167647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5169457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5171307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5173105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5174930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5176738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5178560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5180403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5182292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5184092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5185934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5187775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5189614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5191434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5193249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5195055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5196847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5198670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5200497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5202292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5204136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5206016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5207840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5209644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5211491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5213373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5215191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5217006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5218819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5220609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5222433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5224217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5226015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5227815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5229691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5231534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5233336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5235183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5237070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5238851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5240667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5242483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5244313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5246103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5247907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5249736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5251553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5253426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5255329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5257156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5259432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5261445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5263251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5265066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5266871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5268675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5270458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5272279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5274093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5275895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5277756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5279624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5281448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5283269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5285129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5286925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5288757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5290544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5292346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5294151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5295981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5297773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5299581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5301422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5303307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5305089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5306904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5308740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5310587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5312373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5314156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5315970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5317775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5319576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5321372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5323181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5325032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5326880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5328667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5330488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5332353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5334220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5336008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5337819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5339662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5341449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5343270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5345081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5346915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5348750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5350627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5352431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5354244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5356107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5357978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5360167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5362030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5363820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5365631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5367432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5369243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5371048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5372839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_587_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5374743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5376643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5378446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5380299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 80%] 2024-08-08T21:00:14.5382195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5384014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5385835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5387641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5389469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5391287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5393103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5394909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5396730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5398576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5400412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5402216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5404056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5405940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5407726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5409544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5411348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5413177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5414958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5416776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5418584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5420411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5422249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5424114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5425924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5427809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5429665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5431463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5433284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5435116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5436926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5438727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5440546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5442357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5444154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5445993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5447875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5449686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5451489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5453328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5455231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5457054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5459213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5461069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5462866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5464699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5466491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5468300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5470192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5472097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5473880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5475695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5477556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5479456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5481241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5483060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5484920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5486750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5488451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5490237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5492049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5493899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5495747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5497538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5499355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5501200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5503049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5504833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5506659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5508480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5510283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5512080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5513906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5515730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5517517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5519381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5521254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5523085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5524927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5539185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5541097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5542921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5544711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5546513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5548307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5550138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5551925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5553726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5555683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5557586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5559868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5561814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5563719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5565532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5567330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5569128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5570949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5572758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5574564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5576374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5578187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5580063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5581938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5583735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5585585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5587434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5589227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5591009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5592807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5594640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5596418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5598225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5600030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5601853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5603674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5605529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5607313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5609135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5610953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5612801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5614604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5616432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5618223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5620009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5621853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5623673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5625467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5627312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5629165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5630964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5632745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5634559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5636413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5638210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5640009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5641790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5643596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5645396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5647182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5648959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5650751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5652622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5654453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5656294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5658193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5660483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5662275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5664086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5665897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5667719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5669497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5671310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5673102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5674903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5676765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5678626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5680437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5682303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5684166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5685953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5687763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5689571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5691372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5693161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5694967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5696778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5698569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5700392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5702256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5704064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5705898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5707760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5709550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5711372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5713153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5714956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5716739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5718554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5720316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5722098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5723926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5725801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5727567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5729362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5731199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5733055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5734826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5736632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5738427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5740227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5742027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5743820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5745630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5747477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5749326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5751116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5752912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5754763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5756653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5758769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5760630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5762427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5764206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5765978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5767774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5769587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5771434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5773316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5775100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5776912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5778745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5780622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5782407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5784223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5786001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5787802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5789590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5791414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5793192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5794991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5796823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5798680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5800470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5802309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5804154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5805943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5807724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5809502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5811301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5813087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5814873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5816657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5818461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5820296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5822135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5823910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5825732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5827591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5829349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5831139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5832925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5834741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5836509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5838311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5840099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5841913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5843729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5845582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5847356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5849160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5850965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5852800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5854574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5856412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5858206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5860417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5862224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5864024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5865806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5867662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5869550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5871356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5873149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5874993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5876880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5878750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5880582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5882289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5884080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5885905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5887689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5889498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5891271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5893124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5894962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5896743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5898569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5900433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5902195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5903992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5905782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5907596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5909366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5911167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5912962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5914761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5916599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5918438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5920250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5922100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5923951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5925737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5927545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5929360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5931163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5932950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5934745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5936537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5938322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5940139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5941977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5943790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5945567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5947406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5949242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5951048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5952826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5954631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_128_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5956457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5958288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5960497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5962297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5964177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5966082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5967850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5969651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5971519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5973403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5975196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5976982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5978782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5980567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5982351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5984127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5985931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5987784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5989627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5991410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5993209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5995041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5996881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.5998656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_160_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6000437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6002252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6004016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6005811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6007609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6009426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6011190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6013047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6014893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6016702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6018518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6020360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6022134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6023938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6025705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6027487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6029265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6031064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6032855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6034628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6036473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6038943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6040723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6042539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6044398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6046196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6047989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6049772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6051587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6053388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6055185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6057030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6059214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6061183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6063046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6064851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6066638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6068515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6070363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6072157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6073956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6075784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6077553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6079352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6081146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6082960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6084766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6086624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_192_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6088418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6090227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6092154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6093901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6095714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6097527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6099338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6101131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6102944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6104757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6106548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6108324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6110169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6112015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6113799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6115618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6117476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6119268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6121044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6122848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6124645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6126459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6128237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6130036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_203_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6131819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6133680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6135501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6137302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6139135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6141002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6142767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6144570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6146362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6148170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6149963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6151747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6153547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6155381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6157224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6159433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6161311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6163114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6165002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6166849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6168646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6170441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6172233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6174001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_21_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6175801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6177622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6179395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6181258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6183124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6184944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6186715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6188570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6190410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6192232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6194016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6195818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6197596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6199406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6201180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6202958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6204782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6206637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6208427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6210206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6212057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6213909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6215694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6217476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_256_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6219282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6221074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 81%] 2024-08-08T21:00:14.6222857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6224637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6226451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6228244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6230075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6231909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6233695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6235554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6237373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6239166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6240934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6242744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6244513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6246298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6248079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6249898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6251659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6253489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6255376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6257184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6259434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6261334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6263115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6264945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6266707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6270211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6273707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6277147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6280628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6284099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6287699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6291232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6294690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6298135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6301627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6305101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6308608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6311956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6315413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6318858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6322300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6325713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6329165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6332671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6336182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6339614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6343061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6346574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6350057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6353491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6356997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6360905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6364379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6367832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6371282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6374753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6378206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6381745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6385279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6388721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6392221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6395699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6399131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6402599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6406053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6409491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6412941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6416389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6419847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6423274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_72_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6426786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6430295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6433728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6437204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6440691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6444144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6447587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6451022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6454479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6457995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6461897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6465424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6468773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6472300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6475798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6479277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6482635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6486142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6489650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6493073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6496528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6499988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6503414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6506824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6510282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6513744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6517252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6520732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6524195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6527710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6531162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6534673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6538136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6541626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6545094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6548535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_False_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6551975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6555463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6559303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6562755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6566337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6569890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6573330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6576808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6580352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale0_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6583814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_bfloat16_scale_l1_cuda_bfloat16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6587264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6590696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_96_is_causal_True_dropout_p_0_48_float16_scale_l1_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6594322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6598057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6601803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6605541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6609275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6613052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6616853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6620600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6624379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6628140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6631895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6635638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6639379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6643094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6646828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6650577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6654315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6658121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6662357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6666112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6669944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6673754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6677488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6681228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6684943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6688657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6692390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6696122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6699831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6703613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6707406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6711142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6714906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6718675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6722406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6726152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6729890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6733603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6737328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6741123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6744775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6748533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6752322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6756093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6760214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6764068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6767861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6771586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6775315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6779042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6782764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6786501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6790231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6793940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6797728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6801550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6805261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6809012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6812781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6816494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6820210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6823902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6827616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6831338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_32_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6835067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6838789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6842576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6846369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6850104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6853843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6857680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6861843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6865584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6869306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6873027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6876755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6880468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6884180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6887979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6891789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6895503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6899212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6903007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6906789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6910505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6914209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6917942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6921681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6925399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6929104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6932816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6936567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6940343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6944048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6947807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6951580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_256_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6955287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6959411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6963159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6966887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6970605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6974327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6978039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6981854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6985659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6989346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6993137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.6996923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7000632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7004342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7008061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7011790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_256_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7015497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7019192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0003s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7022902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7026670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7030439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7034137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7037848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7041623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_64_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7045366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7049049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7052746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7056517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_0_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7060644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7064364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale0_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7068079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_False_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7071761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_flash_attention_vs_math_ref_grads_nestedtensor_batch_size_8_max_seq_len_q_32_max_seq_len_kv_32_head_dim_8_dropout_p_0_1_float16_scale_l1_is_causal_True_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7075511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7079213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7082816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7086476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7090156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7093762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7097349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7100949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths0_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7104536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7108146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7111751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7115353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7118952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7122605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7126258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7129851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths1_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7133504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7137157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7140745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7144338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7147929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7151531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7155129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7159156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_1_sequence_legnths2_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7162796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7166404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7170102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7173766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7177363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7181038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7184716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7188304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths0_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7191902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7195525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7199094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7202691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7206299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7209908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7213553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7217203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths1_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7220787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_32_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7224390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_32_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7228039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_32_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7231680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_32_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7235274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_64_is_causal_False_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7238878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_64_is_causal_False_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7242476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_64_is_causal_True_dropout_p_0_0_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7246052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_attention_vs_math_ref_grads_cudagraph_batch_size_8_sequence_legnths2_head_dim_64_is_causal_True_dropout_p_0_22_float16_fused_kernel0_cuda_float16 SKIPPED [0.0002s] (Does not support SDPA or pre-SM80 hardware) [ 82%] 2024-08-08T21:00:14.7248905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_backwards_throws_determinism_warning_fused_kernel0_warn_only_False_cuda PASSED [0.0042s] [ 82%] 2024-08-08T21:00:14.7251125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_backwards_throws_determinism_warning_fused_kernel0_warn_only_True_cuda PASSED [0.0075s] [ 82%] 2024-08-08T21:00:14.7253996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0243s] [ 82%] 2024-08-08T21:00:14.7257472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0248s] [ 82%] 2024-08-08T21:00:14.7261368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0236s] [ 82%] 2024-08-08T21:00:14.7264853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0254s] [ 82%] 2024-08-08T21:00:14.7268275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0241s] [ 82%] 2024-08-08T21:00:14.7271689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0254s] [ 82%] 2024-08-08T21:00:14.7275162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0237s] [ 82%] 2024-08-08T21:00:14.7278704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0217s] [ 82%] 2024-08-08T21:00:14.7282109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0228s] [ 82%] 2024-08-08T21:00:14.7285501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0220s] [ 82%] 2024-08-08T21:00:14.7288916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0239s] [ 82%] 2024-08-08T21:00:14.7292308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0241s] [ 82%] 2024-08-08T21:00:14.7295713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0229s] [ 82%] 2024-08-08T21:00:14.7299115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0239s] [ 82%] 2024-08-08T21:00:14.7302498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0239s] [ 82%] 2024-08-08T21:00:14.7305851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0195s] [ 82%] 2024-08-08T21:00:14.7309306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0222s] [ 82%] 2024-08-08T21:00:14.7312777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0217s] [ 82%] 2024-08-08T21:00:14.7316182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0222s] [ 82%] 2024-08-08T21:00:14.7319629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0236s] [ 82%] 2024-08-08T21:00:14.7323084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0217s] [ 82%] 2024-08-08T21:00:14.7326480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0232s] [ 82%] 2024-08-08T21:00:14.7329846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0231s] [ 82%] 2024-08-08T21:00:14.7333232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0183s] [ 82%] 2024-08-08T21:00:14.7336633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0237s] [ 82%] 2024-08-08T21:00:14.7340030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0217s] [ 82%] 2024-08-08T21:00:14.7343434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0213s] [ 82%] 2024-08-08T21:00:14.7346833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0216s] [ 82%] 2024-08-08T21:00:14.7350209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0212s] [ 82%] 2024-08-08T21:00:14.7353645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0219s] [ 82%] 2024-08-08T21:00:14.7357119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0221s] [ 82%] 2024-08-08T21:00:14.7360930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_False_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0177s] [ 82%] 2024-08-08T21:00:14.7364343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0205s] [ 82%] 2024-08-08T21:00:14.7367843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0213s] [ 82%] 2024-08-08T21:00:14.7371323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0215s] [ 82%] 2024-08-08T21:00:14.7374717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0221s] [ 82%] 2024-08-08T21:00:14.7378137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0209s] [ 82%] 2024-08-08T21:00:14.7381548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0220s] [ 82%] 2024-08-08T21:00:14.7384945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0216s] [ 82%] 2024-08-08T21:00:14.7388329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0179s] [ 82%] 2024-08-08T21:00:14.7391727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0193s] [ 82%] 2024-08-08T21:00:14.7395107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0206s] [ 82%] 2024-08-08T21:00:14.7398493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0210s] [ 82%] 2024-08-08T21:00:14.7401947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0216s] [ 82%] 2024-08-08T21:00:14.7405406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0210s] [ 82%] 2024-08-08T21:00:14.7408791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0218s] [ 82%] 2024-08-08T21:00:14.7412221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0212s] [ 82%] 2024-08-08T21:00:14.7415634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_False_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0177s] [ 82%] 2024-08-08T21:00:14.7419024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0204s] [ 82%] 2024-08-08T21:00:14.7422420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0210s] [ 82%] 2024-08-08T21:00:14.7425817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0203s] [ 82%] 2024-08-08T21:00:14.7429212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0209s] [ 82%] 2024-08-08T21:00:14.7432602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0204s] [ 82%] 2024-08-08T21:00:14.7435971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0216s] [ 82%] 2024-08-08T21:00:14.7439350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0213s] [ 82%] 2024-08-08T21:00:14.7442721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_False_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0174s] [ 82%] 2024-08-08T21:00:14.7446169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0035s] [ 82%] 2024-08-08T21:00:14.7449614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0036s] [ 82%] 2024-08-08T21:00:14.7452995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0035s] [ 82%] 2024-08-08T21:00:14.7456400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_False_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0036s] [ 82%] 2024-08-08T21:00:14.7460212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_False_cuda PASSED [0.0035s] [ 82%] 2024-08-08T21:00:14.7463690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_False_expand_v_num_heads_True_cuda PASSED [0.0036s] [ 82%] 2024-08-08T21:00:14.7467056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_False_cuda PASSED [0.0037s] [ 82%] 2024-08-08T21:00:14.7470426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_kernel0_expand_q_batch_True_expand_k_batch_True_expand_v_batch_True_expand_q_num_heads_True_expand_k_num_heads_True_expand_v_num_heads_True_cuda PASSED [0.0034s] [ 82%] 2024-08-08T21:00:14.7473002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_nested_broadcasting_query_dense_cuda PASSED [0.0310s] [ 82%] 2024-08-08T21:00:14.7474860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_kernels_seq_len_1_inputs_fused_kernel0_cuda PASSED [0.0301s] [ 82%] 2024-08-08T21:00:14.7476612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_sdp_choice_type_dense_cuda PASSED [0.0017s] [ 82%] 2024-08-08T21:00:14.7478276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_fused_sdp_choice_type_nested_cuda PASSED [0.0020s] [ 82%] 2024-08-08T21:00:14.7480114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_long_sequence_mask_float16_cuda_float16 PASSED [0.0013s] [ 82%] 2024-08-08T21:00:14.7482055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_long_sequence_mask_float32_cuda_float32 PASSED [0.0013s] [ 82%] 2024-08-08T21:00:14.7483928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_non_contig_mask_bug_cuda PASSED [0.0018s] [ 82%] 2024-08-08T21:00:14.7485814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_non_contiguous_mask_float16_cuda_float16 PASSED [0.0020s] [ 82%] 2024-08-08T21:00:14.7487767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_non_contiguous_mask_float32_cuda_float32 PASSED [0.0033s] [ 82%] 2024-08-08T21:00:14.7489652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_pad_mask_float16_cuda_float16 PASSED [0.0020s] [ 82%] 2024-08-08T21:00:14.7491480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_attention_pad_mask_float32_cuda_float32 PASSED [0.0020s] [ 82%] 2024-08-08T21:00:14.7493853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_eff_backwards_determinism_cuda SKIPPED [0.0003s] (This test is not behaving deterministaclly non-deterministaclly on CI/CD) [ 82%] 2024-08-08T21:00:14.7496874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0188s] [ 82%] 2024-08-08T21:00:14.7499990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 82%] 2024-08-08T21:00:14.7503089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0082s] [ 82%] 2024-08-08T21:00:14.7508456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 SKIPPED [0.0006s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131105 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 82%] 2024-08-08T21:00:14.7515689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131126 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 82%] 2024-08-08T21:00:14.7522861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131168 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 82%] 2024-08-08T21:00:14.7528152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 [W808 20:57:12.551227631 attention.cpp:807] Warning: Dropout mask should only be used for testing purposes. (function operator()) 2024-08-08T21:00:14.7530551Z ('RERUN', {'yellow': True}) [0.0071s] [ 82%] 2024-08-08T21:00:14.7532670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 ('RERUN', {'yellow': True}) [0.0062s] [ 82%] 2024-08-08T21:00:14.7535857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 FAILED [0.0055s] [ 82%] 2024-08-08T21:00:14.7537385Z 2024-08-08T21:00:14.7537645Z ==================================== RERUNS ==================================== 2024-08-08T21:00:14.7539442Z _ TestSDPACudaOnlyCUDA.test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 _ 2024-08-08T21:00:14.7540943Z Traceback (most recent call last): 2024-08-08T21:00:14.7542091Z File "/var/lib/jenkins/workspace/test/test_transformers.py", line 2930, in test_mem_efficient_attention_attn_mask_vs_math_ref_grads 2024-08-08T21:00:14.7543210Z check_out_and_grad( 2024-08-08T21:00:14.7544021Z File "/var/lib/jenkins/workspace/test/test_transformers.py", line 184, in check_out_and_grad 2024-08-08T21:00:14.7545285Z _check_equal(ref_grad, lp_ref_grad, comp_grad, fudge_factors.get(name, default_fudge_factor), name) 2024-08-08T21:00:14.7546513Z File "/var/lib/jenkins/workspace/test/test_transformers.py", line 144, in _check_equal 2024-08-08T21:00:14.7547407Z raise ValueError(msg) 2024-08-08T21:00:14.7548213Z ValueError: grad_query Test error 0.6408719430776678 is greater than threshold 1e-05! 2024-08-08T21:00:14.7548865Z 2024-08-08T21:00:14.7549212Z To execute this test, run the following from the base repo dir: 2024-08-08T21:00:14.7551255Z python test/test_transformers.py -k TestSDPACudaOnlyCUDA.test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 2024-08-08T21:00:14.7552756Z 2024-08-08T21:00:14.7553167Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-08-08T21:00:14.7555080Z _ TestSDPACudaOnlyCUDA.test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 _ 2024-08-08T21:00:14.7556574Z Traceback (most recent call last): 2024-08-08T21:00:14.7557701Z File "/var/lib/jenkins/workspace/test/test_transformers.py", line 2930, in test_mem_efficient_attention_attn_mask_vs_math_ref_grads 2024-08-08T21:00:14.7559213Z check_out_and_grad( 2024-08-08T21:00:14.7560024Z File "/var/lib/jenkins/workspace/test/test_transformers.py", line 184, in check_out_and_grad 2024-08-08T21:00:14.7561299Z _check_equal(ref_grad, lp_ref_grad, comp_grad, fudge_factors.get(name, default_fudge_factor), name) 2024-08-08T21:00:14.7562520Z File "/var/lib/jenkins/workspace/test/test_transformers.py", line 144, in _check_equal 2024-08-08T21:00:14.7563382Z raise ValueError(msg) 2024-08-08T21:00:14.7564247Z ValueError: grad_query Test error 0.6408719430776678 is greater than threshold 1e-05! 2024-08-08T21:00:14.7564896Z 2024-08-08T21:00:14.7565227Z To execute this test, run the following from the base repo dir: 2024-08-08T21:00:14.7567234Z python test/test_transformers.py -k TestSDPACudaOnlyCUDA.test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 2024-08-08T21:00:14.7568692Z 2024-08-08T21:00:14.7569100Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-08-08T21:00:14.7569943Z =================================== FAILURES =================================== 2024-08-08T21:00:14.7571695Z _ TestSDPACudaOnlyCUDA.test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 _ 2024-08-08T21:00:14.7573158Z Traceback (most recent call last): 2024-08-08T21:00:14.7574303Z File "/var/lib/jenkins/workspace/test/test_transformers.py", line 2930, in test_mem_efficient_attention_attn_mask_vs_math_ref_grads 2024-08-08T21:00:14.7575418Z check_out_and_grad( 2024-08-08T21:00:14.7576230Z File "/var/lib/jenkins/workspace/test/test_transformers.py", line 184, in check_out_and_grad 2024-08-08T21:00:14.7577493Z _check_equal(ref_grad, lp_ref_grad, comp_grad, fudge_factors.get(name, default_fudge_factor), name) 2024-08-08T21:00:14.7578722Z File "/var/lib/jenkins/workspace/test/test_transformers.py", line 144, in _check_equal 2024-08-08T21:00:14.7579563Z raise ValueError(msg) 2024-08-08T21:00:14.7580507Z ValueError: grad_query Test error 0.6408719430776678 is greater than threshold 1e-05! 2024-08-08T21:00:14.7581239Z 2024-08-08T21:00:14.7581594Z To execute this test, run the following from the base repo dir: 2024-08-08T21:00:14.7583589Z python test/test_transformers.py -k TestSDPACudaOnlyCUDA.test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 2024-08-08T21:00:14.7585041Z 2024-08-08T21:00:14.7585451Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-08-08T21:00:14.7587122Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_transformers/test_transformers-fc7fcbdc7cb25791.xml - 2024-08-08T21:00:14.7588677Z =========================== short test summary info ============================ 2024-08-08T21:00:14.7591819Z FAILED [0.0055s] test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 - ValueError: grad_query Test error 0.6408719430776678 is greater than threshold 1e-05! 2024-08-08T21:00:14.7593894Z 2024-08-08T21:00:14.7594226Z To execute this test, run the following from the base repo dir: 2024-08-08T21:00:14.7596244Z python test/test_transformers.py -k TestSDPACudaOnlyCUDA.test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 2024-08-08T21:00:14.7597699Z 2024-08-08T21:00:14.7598110Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-08-08T21:00:14.7599157Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-08-08T21:00:14.7600499Z ======= 1 failed, 178 passed, 37382 skipped, 2 rerun in 91.42s (0:01:31) ======= 2024-08-08T21:00:14.7601499Z Got exit code 1 2024-08-08T21:00:14.7601888Z Retrying single test... 2024-08-08T21:00:14.7602974Z Test results will be stored in test-reports/python-pytest/test_transformers/test_transformers-1f9fa027e9315f70.xml 2024-08-08T21:00:14.7604257Z ============================= test session starts ============================== 2024-08-08T21:00:14.7605423Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-08T21:00:14.7606294Z cachedir: .pytest_cache 2024-08-08T21:00:14.7607433Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-08T21:00:14.7608520Z rootdir: /var/lib/jenkins/workspace 2024-08-08T21:00:14.7609016Z configfile: pytest.ini 2024-08-08T21:00:14.7609991Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-08T21:00:14.7611319Z collecting ... collected 45344 items / 45343 deselected / 1 selected 2024-08-08T21:00:14.7613401Z stepcurrent: skipping 37560 already run items. Running only test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 2024-08-08T21:00:14.7615292Z Running 1 items in this shard 2024-08-08T21:00:14.7615622Z 2024-08-08T21:00:14.7617528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 [W808 20:57:33.633759513 attention.cpp:807] Warning: Dropout mask should only be used for testing purposes. (function operator()) 2024-08-08T21:00:14.7619845Z PASSED [0.2166s] [100%] 2024-08-08T21:00:14.7620212Z 2024-08-08T21:00:14.7621347Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_transformers/test_transformers-1f9fa027e9315f70.xml - 2024-08-08T21:00:14.7623178Z ===================== 1 passed, 45343 deselected in 4.81s ====================== 2024-08-08T21:00:14.7624017Z Got exit code 0 2024-08-08T21:00:14.7624622Z Test succeeeded in new process, continuing with the rest of the tests 2024-08-08T21:00:14.7626009Z Test results will be stored in test-reports/python-pytest/test_transformers/test_transformers-6f63f59cbd7291bb.xml 2024-08-08T21:00:14.7627277Z ============================= test session starts ============================== 2024-08-08T21:00:14.7628440Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-08T21:00:14.7629331Z cachedir: .pytest_cache 2024-08-08T21:00:14.7630455Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-08T21:00:14.7631538Z rootdir: /var/lib/jenkins/workspace 2024-08-08T21:00:14.7632042Z configfile: pytest.ini 2024-08-08T21:00:14.7633069Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-08T21:00:14.7634389Z collecting ... collected 45344 items / 37561 deselected / 7783 selected 2024-08-08T21:00:14.7635204Z stepcurrent: skipping 37561 already run items. 2024-08-08T21:00:14.7635784Z Running 7783 items in this shard 2024-08-08T21:00:14.7636113Z 2024-08-08T21:00:14.7639904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 SKIPPED [0.0009s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131383 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7644936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.1297s] [ 0%] 2024-08-08T21:00:14.7650224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 SKIPPED [0.0008s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131435 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7657429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131090 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7665049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131139 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7672306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0009s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131495 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7679633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131609 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7686891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131624 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7694054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131537 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7701217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131181 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7706219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0121s] [ 0%] 2024-08-08T21:00:14.7711469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131605 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7718697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 SKIPPED [0.0008s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131606 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7725898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131602 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7733110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131633 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7740316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 SKIPPED [0.0007s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/131646 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 0%] 2024-08-08T21:00:14.7745619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 [W808 20:57:51.098775862 attention.cpp:807] Warning: Dropout mask should only be used for testing purposes. (function operator()) 2024-08-08T21:00:14.7747928Z PASSED [0.0965s] [ 0%] 2024-08-08T21:00:14.7749888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0094s] [ 0%] 2024-08-08T21:00:14.7752999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 0%] 2024-08-08T21:00:14.7756156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0086s] [ 0%] 2024-08-08T21:00:14.7759699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 0%] 2024-08-08T21:00:14.7762843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0137s] [ 0%] 2024-08-08T21:00:14.7766031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0126s] [ 0%] 2024-08-08T21:00:14.7769231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0110s] [ 0%] 2024-08-08T21:00:14.7772354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 0%] 2024-08-08T21:00:14.7775467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0129s] [ 0%] 2024-08-08T21:00:14.7778650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0135s] [ 0%] 2024-08-08T21:00:14.7781825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0082s] [ 0%] 2024-08-08T21:00:14.7784941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 0%] 2024-08-08T21:00:14.7788050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0177s] [ 0%] 2024-08-08T21:00:14.7791191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0184s] [ 0%] 2024-08-08T21:00:14.7794323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0108s] [ 0%] 2024-08-08T21:00:14.7797442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0110s] [ 0%] 2024-08-08T21:00:14.7800539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0129s] [ 0%] 2024-08-08T21:00:14.7803661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0138s] [ 0%] 2024-08-08T21:00:14.7806771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 0%] 2024-08-08T21:00:14.7809892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 0%] 2024-08-08T21:00:14.7813087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0194s] [ 0%] 2024-08-08T21:00:14.7816276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0169s] [ 0%] 2024-08-08T21:00:14.7819387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0105s] [ 0%] 2024-08-08T21:00:14.7822515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0104s] [ 0%] 2024-08-08T21:00:14.7825683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0127s] [ 0%] 2024-08-08T21:00:14.7828855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0138s] [ 0%] 2024-08-08T21:00:14.7831968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0087s] [ 0%] 2024-08-08T21:00:14.7835090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 0%] 2024-08-08T21:00:14.7838200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0181s] [ 0%] 2024-08-08T21:00:14.7841337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0170s] [ 0%] 2024-08-08T21:00:14.7844481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0114s] [ 0%] 2024-08-08T21:00:14.7847610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0111s] [ 0%] 2024-08-08T21:00:14.7850734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0113s] [ 0%] 2024-08-08T21:00:14.7853839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0107s] [ 0%] 2024-08-08T21:00:14.7857020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0068s] [ 0%] 2024-08-08T21:00:14.7860711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 0%] 2024-08-08T21:00:14.7863830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0147s] [ 0%] 2024-08-08T21:00:14.7866935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0148s] [ 0%] 2024-08-08T21:00:14.7870146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 0%] 2024-08-08T21:00:14.7873344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0094s] [ 0%] 2024-08-08T21:00:14.7876430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0111s] [ 0%] 2024-08-08T21:00:14.7879530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 0%] 2024-08-08T21:00:14.7882654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 0%] 2024-08-08T21:00:14.7885767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 0%] 2024-08-08T21:00:14.7888878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0079s] [ 0%] 2024-08-08T21:00:14.7891994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 0%] 2024-08-08T21:00:14.7895094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 0%] 2024-08-08T21:00:14.7898203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 0%] 2024-08-08T21:00:14.7901302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 0%] 2024-08-08T21:00:14.7904476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 0%] 2024-08-08T21:00:14.7906139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 0%] 2024-08-08T21:00:14.7907751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 0%] 2024-08-08T21:00:14.7909350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 0%] 2024-08-08T21:00:14.7911032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 0%] 2024-08-08T21:00:14.7912677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 0%] 2024-08-08T21:00:14.7914297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 0%] 2024-08-08T21:00:14.7915887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0076s] [ 0%] 2024-08-08T21:00:14.7917495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 0%] 2024-08-08T21:00:14.7919097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 0%] 2024-08-08T21:00:14.7920688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 0%] 2024-08-08T21:00:14.7922301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0080s] [ 1%] 2024-08-08T21:00:14.7923911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 1%] 2024-08-08T21:00:14.7925515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 1%] 2024-08-08T21:00:14.7927168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 1%] 2024-08-08T21:00:14.7928819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 1%] 2024-08-08T21:00:14.7930404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 1%] 2024-08-08T21:00:14.7931990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 1%] 2024-08-08T21:00:14.7933614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 1%] 2024-08-08T21:00:14.7935265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 1%] 2024-08-08T21:00:14.7936873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 1%] 2024-08-08T21:00:14.7938460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 1%] 2024-08-08T21:00:14.7940080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 1%] 2024-08-08T21:00:14.7941668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 1%] 2024-08-08T21:00:14.7943273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 1%] 2024-08-08T21:00:14.7944853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 1%] 2024-08-08T21:00:14.7946462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 1%] 2024-08-08T21:00:14.7948040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 1%] 2024-08-08T21:00:14.7949649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 1%] 2024-08-08T21:00:14.7951281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 1%] 2024-08-08T21:00:14.7952933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 1%] 2024-08-08T21:00:14.7954526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 1%] 2024-08-08T21:00:14.7956159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 1%] 2024-08-08T21:00:14.7957857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 1%] 2024-08-08T21:00:14.7959927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 1%] 2024-08-08T21:00:14.7961560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 1%] 2024-08-08T21:00:14.7963154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 1%] 2024-08-08T21:00:14.7964770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 1%] 2024-08-08T21:00:14.7966361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 1%] 2024-08-08T21:00:14.7967954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 1%] 2024-08-08T21:00:14.7969546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 1%] 2024-08-08T21:00:14.7971141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 1%] 2024-08-08T21:00:14.7972720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 1%] 2024-08-08T21:00:14.7974383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 1%] 2024-08-08T21:00:14.7976071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 1%] 2024-08-08T21:00:14.7977652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 1%] 2024-08-08T21:00:14.7979268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 1%] 2024-08-08T21:00:14.7980902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 1%] 2024-08-08T21:00:14.7982573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 1%] 2024-08-08T21:00:14.7984140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 1%] 2024-08-08T21:00:14.7985734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 1%] 2024-08-08T21:00:14.7987313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 1%] 2024-08-08T21:00:14.7988919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 1%] 2024-08-08T21:00:14.7990485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 1%] 2024-08-08T21:00:14.7992070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 1%] 2024-08-08T21:00:14.7993688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0159s] [ 1%] 2024-08-08T21:00:14.7995291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0175s] [ 1%] 2024-08-08T21:00:14.7996897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0086s] [ 1%] 2024-08-08T21:00:14.7998555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 1%] 2024-08-08T21:00:14.8000227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0232s] [ 1%] 2024-08-08T21:00:14.8001832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0240s] [ 1%] 2024-08-08T21:00:14.8003449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0122s] [ 1%] 2024-08-08T21:00:14.8005107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0123s] [ 1%] 2024-08-08T21:00:14.8006766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0164s] [ 1%] 2024-08-08T21:00:14.8008365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0183s] [ 1%] 2024-08-08T21:00:14.8009960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0087s] [ 1%] 2024-08-08T21:00:14.8011584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 1%] 2024-08-08T21:00:14.8013176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0232s] [ 1%] 2024-08-08T21:00:14.8014799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0240s] [ 1%] 2024-08-08T21:00:14.8016404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0124s] [ 1%] 2024-08-08T21:00:14.8018042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0123s] [ 1%] 2024-08-08T21:00:14.8019625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0178s] [ 1%] 2024-08-08T21:00:14.8021288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0198s] [ 1%] 2024-08-08T21:00:14.8022925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0090s] [ 1%] 2024-08-08T21:00:14.8024545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 1%] 2024-08-08T21:00:14.8026134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0247s] [ 1%] 2024-08-08T21:00:14.8027798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0252s] [ 1%] 2024-08-08T21:00:14.8030282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0145s] [ 1%] 2024-08-08T21:00:14.8031886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0145s] [ 1%] 2024-08-08T21:00:14.8033488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0153s] [ 1%] 2024-08-08T21:00:14.8035090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0165s] [ 1%] 2024-08-08T21:00:14.8036700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0082s] [ 1%] 2024-08-08T21:00:14.8038287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 1%] 2024-08-08T21:00:14.8039902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0227s] [ 1%] 2024-08-08T21:00:14.8041510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0236s] [ 1%] 2024-08-08T21:00:14.8043118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0120s] [ 1%] 2024-08-08T21:00:14.8044712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0120s] [ 1%] 2024-08-08T21:00:14.8046346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 1%] 2024-08-08T21:00:14.8048015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 1%] 2024-08-08T21:00:14.8049596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8051249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8052889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 2%] 2024-08-08T21:00:14.8054504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 2%] 2024-08-08T21:00:14.8056146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 2%] 2024-08-08T21:00:14.8057776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 2%] 2024-08-08T21:00:14.8059732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8061374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 2%] 2024-08-08T21:00:14.8062963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 2%] 2024-08-08T21:00:14.8064570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 2%] 2024-08-08T21:00:14.8066177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 2%] 2024-08-08T21:00:14.8067779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 2%] 2024-08-08T21:00:14.8069475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 2%] 2024-08-08T21:00:14.8071148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 2%] 2024-08-08T21:00:14.8072747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 2%] 2024-08-08T21:00:14.8074345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 2%] 2024-08-08T21:00:14.8076010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 2%] 2024-08-08T21:00:14.8077678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8079286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 2%] 2024-08-08T21:00:14.8080900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 2%] 2024-08-08T21:00:14.8082525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 2%] 2024-08-08T21:00:14.8084121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 2%] 2024-08-08T21:00:14.8085698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8087308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 2%] 2024-08-08T21:00:14.8088893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 2%] 2024-08-08T21:00:14.8090492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 2%] 2024-08-08T21:00:14.8092119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 2%] 2024-08-08T21:00:14.8093791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 2%] 2024-08-08T21:00:14.8095382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 2%] 2024-08-08T21:00:14.8097000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 2%] 2024-08-08T21:00:14.8098667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 2%] 2024-08-08T21:00:14.8100296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8101866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 2%] 2024-08-08T21:00:14.8103451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8105053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 2%] 2024-08-08T21:00:14.8106665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 2%] 2024-08-08T21:00:14.8108277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 2%] 2024-08-08T21:00:14.8109892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 2%] 2024-08-08T21:00:14.8111490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 2%] 2024-08-08T21:00:14.8113081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 2%] 2024-08-08T21:00:14.8114667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8116291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 2%] 2024-08-08T21:00:14.8117961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 2%] 2024-08-08T21:00:14.8119554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 2%] 2024-08-08T21:00:14.8121139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 2%] 2024-08-08T21:00:14.8122802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 2%] 2024-08-08T21:00:14.8124432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8126034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 2%] 2024-08-08T21:00:14.8127612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 2%] 2024-08-08T21:00:14.8129225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 2%] 2024-08-08T21:00:14.8130813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 2%] 2024-08-08T21:00:14.8132422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 2%] 2024-08-08T21:00:14.8134014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 2%] 2024-08-08T21:00:14.8135638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 2%] 2024-08-08T21:00:14.8137209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 2%] 2024-08-08T21:00:14.8138834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 2%] 2024-08-08T21:00:14.8140475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 2%] 2024-08-08T21:00:14.8142067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 2%] 2024-08-08T21:00:14.8143661Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 2%] 2024-08-08T21:00:14.8145292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 2%] 2024-08-08T21:00:14.8146935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 2%] 2024-08-08T21:00:14.8148516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 2%] 2024-08-08T21:00:14.8150124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 2%] 2024-08-08T21:00:14.8151729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 2%] 2024-08-08T21:00:14.8153352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 2%] 2024-08-08T21:00:14.8154961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 2%] 2024-08-08T21:00:14.8156615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0105s] [ 2%] 2024-08-08T21:00:14.8158256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0107s] [ 2%] 2024-08-08T21:00:14.8160227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 2%] 2024-08-08T21:00:14.8161859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 2%] 2024-08-08T21:00:14.8163534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0080s] [ 2%] 2024-08-08T21:00:14.8165241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0088s] [ 2%] 2024-08-08T21:00:14.8166830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 2%] 2024-08-08T21:00:14.8168454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 2%] 2024-08-08T21:00:14.8170107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0109s] [ 2%] 2024-08-08T21:00:14.8171792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0109s] [ 2%] 2024-08-08T21:00:14.8173384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 2%] 2024-08-08T21:00:14.8174999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 2%] 2024-08-08T21:00:14.8176613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0088s] [ 3%] 2024-08-08T21:00:14.8178212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0093s] [ 3%] 2024-08-08T21:00:14.8179812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 3%] 2024-08-08T21:00:14.8181422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 3%] 2024-08-08T21:00:14.8183044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0117s] [ 3%] 2024-08-08T21:00:14.8184646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0115s] [ 3%] 2024-08-08T21:00:14.8186303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0085s] [ 3%] 2024-08-08T21:00:14.8187960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 3%] 2024-08-08T21:00:14.8189568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0076s] [ 3%] 2024-08-08T21:00:14.8191158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 3%] 2024-08-08T21:00:14.8192809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 3%] 2024-08-08T21:00:14.8194467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 3%] 2024-08-08T21:00:14.8196049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0106s] [ 3%] 2024-08-08T21:00:14.8197659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 3%] 2024-08-08T21:00:14.8199248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 3%] 2024-08-08T21:00:14.8200877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 3%] 2024-08-08T21:00:14.8202464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0116s] [ 3%] 2024-08-08T21:00:14.8204092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0125s] [ 3%] 2024-08-08T21:00:14.8205685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 3%] 2024-08-08T21:00:14.8207304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 3%] 2024-08-08T21:00:14.8208893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0161s] [ 3%] 2024-08-08T21:00:14.8210574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0164s] [ 3%] 2024-08-08T21:00:14.8212220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 3%] 2024-08-08T21:00:14.8213828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 3%] 2024-08-08T21:00:14.8215439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0119s] [ 3%] 2024-08-08T21:00:14.8217088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0131s] [ 3%] 2024-08-08T21:00:14.8218747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 3%] 2024-08-08T21:00:14.8220347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 3%] 2024-08-08T21:00:14.8221957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0166s] [ 3%] 2024-08-08T21:00:14.8223567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0168s] [ 3%] 2024-08-08T21:00:14.8225179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0095s] [ 3%] 2024-08-08T21:00:14.8226781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 3%] 2024-08-08T21:00:14.8228389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0139s] [ 3%] 2024-08-08T21:00:14.8229996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0148s] [ 3%] 2024-08-08T21:00:14.8231589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 3%] 2024-08-08T21:00:14.8233249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 3%] 2024-08-08T21:00:14.8234902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0177s] [ 3%] 2024-08-08T21:00:14.8236525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0180s] [ 3%] 2024-08-08T21:00:14.8238116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0109s] [ 3%] 2024-08-08T21:00:14.8239782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0110s] [ 3%] 2024-08-08T21:00:14.8241422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0109s] [ 3%] 2024-08-08T21:00:14.8243027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0116s] [ 3%] 2024-08-08T21:00:14.8244613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 3%] 2024-08-08T21:00:14.8246234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 3%] 2024-08-08T21:00:14.8247827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0158s] [ 3%] 2024-08-08T21:00:14.8249418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0163s] [ 3%] 2024-08-08T21:00:14.8251026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0091s] [ 3%] 2024-08-08T21:00:14.8252635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0091s] [ 3%] 2024-08-08T21:00:14.8254232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 3%] 2024-08-08T21:00:14.8255863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 3%] 2024-08-08T21:00:14.8257523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 3%] 2024-08-08T21:00:14.8259551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 3%] 2024-08-08T21:00:14.8261224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 3%] 2024-08-08T21:00:14.8262914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 3%] 2024-08-08T21:00:14.8264597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 3%] 2024-08-08T21:00:14.8266193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 3%] 2024-08-08T21:00:14.8267772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 3%] 2024-08-08T21:00:14.8269394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 3%] 2024-08-08T21:00:14.8270991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 3%] 2024-08-08T21:00:14.8272607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 3%] 2024-08-08T21:00:14.8274195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0080s] [ 3%] 2024-08-08T21:00:14.8275823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 3%] 2024-08-08T21:00:14.8277415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 3%] 2024-08-08T21:00:14.8279030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 3%] 2024-08-08T21:00:14.8280673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 3%] 2024-08-08T21:00:14.8282360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 3%] 2024-08-08T21:00:14.8283945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 3%] 2024-08-08T21:00:14.8285535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 3%] 2024-08-08T21:00:14.8287207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0087s] [ 3%] 2024-08-08T21:00:14.8288885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 3%] 2024-08-08T21:00:14.8290487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 3%] 2024-08-08T21:00:14.8292094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 3%] 2024-08-08T21:00:14.8293702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 3%] 2024-08-08T21:00:14.8295289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 3%] 2024-08-08T21:00:14.8296882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 3%] 2024-08-08T21:00:14.8298469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 3%] 2024-08-08T21:00:14.8300074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0079s] [ 3%] 2024-08-08T21:00:14.8301659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 3%] 2024-08-08T21:00:14.8303305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 4%] 2024-08-08T21:00:14.8304952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 4%] 2024-08-08T21:00:14.8306550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 4%] 2024-08-08T21:00:14.8308156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 4%] 2024-08-08T21:00:14.8309773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 4%] 2024-08-08T21:00:14.8311481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 4%] 2024-08-08T21:00:14.8313005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 4%] 2024-08-08T21:00:14.8314616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 4%] 2024-08-08T21:00:14.8316212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 4%] 2024-08-08T21:00:14.8317841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 4%] 2024-08-08T21:00:14.8319421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 4%] 2024-08-08T21:00:14.8321026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 4%] 2024-08-08T21:00:14.8322606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 4%] 2024-08-08T21:00:14.8324189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 4%] 2024-08-08T21:00:14.8325780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 4%] 2024-08-08T21:00:14.8327428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 4%] 2024-08-08T21:00:14.8329140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 4%] 2024-08-08T21:00:14.8330747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 4%] 2024-08-08T21:00:14.8332339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 4%] 2024-08-08T21:00:14.8333975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 4%] 2024-08-08T21:00:14.8335618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 4%] 2024-08-08T21:00:14.8337198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 4%] 2024-08-08T21:00:14.8338806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0081s] [ 4%] 2024-08-08T21:00:14.8340411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 4%] 2024-08-08T21:00:14.8341999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 4%] 2024-08-08T21:00:14.8343618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 4%] 2024-08-08T21:00:14.8345191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 4%] 2024-08-08T21:00:14.8346800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 4%] 2024-08-08T21:00:14.8348365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 4%] 2024-08-08T21:00:14.8349996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 4%] 2024-08-08T21:00:14.8351619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 4%] 2024-08-08T21:00:14.8353225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 4%] 2024-08-08T21:00:14.8354787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 4%] 2024-08-08T21:00:14.8356479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 4%] 2024-08-08T21:00:14.8358133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0197s] [ 4%] 2024-08-08T21:00:14.8360403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0213s] [ 4%] 2024-08-08T21:00:14.8362053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0090s] [ 4%] 2024-08-08T21:00:14.8363670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 4%] 2024-08-08T21:00:14.8365292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0281s] [ 4%] 2024-08-08T21:00:14.8366900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0293s] [ 4%] 2024-08-08T21:00:14.8368516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0137s] [ 4%] 2024-08-08T21:00:14.8370131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 4%] 2024-08-08T21:00:14.8371741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0205s] [ 4%] 2024-08-08T21:00:14.8373334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0229s] [ 4%] 2024-08-08T21:00:14.8375048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0097s] [ 4%] 2024-08-08T21:00:14.8376726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0097s] [ 4%] 2024-08-08T21:00:14.8378318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0287s] [ 4%] 2024-08-08T21:00:14.8379941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0297s] [ 4%] 2024-08-08T21:00:14.8381609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0142s] [ 4%] 2024-08-08T21:00:14.8383306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0142s] [ 4%] 2024-08-08T21:00:14.8384889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0233s] [ 4%] 2024-08-08T21:00:14.8386513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0254s] [ 4%] 2024-08-08T21:00:14.8388107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0102s] [ 4%] 2024-08-08T21:00:14.8389723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0106s] [ 4%] 2024-08-08T21:00:14.8391308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0313s] [ 4%] 2024-08-08T21:00:14.8392938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0322s] [ 4%] 2024-08-08T21:00:14.8394543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0170s] [ 4%] 2024-08-08T21:00:14.8396147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0168s] [ 4%] 2024-08-08T21:00:14.8397790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0188s] [ 4%] 2024-08-08T21:00:14.8399427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0203s] [ 4%] 2024-08-08T21:00:14.8401030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0090s] [ 4%] 2024-08-08T21:00:14.8402627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0091s] [ 4%] 2024-08-08T21:00:14.8404275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0277s] [ 4%] 2024-08-08T21:00:14.8405923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0286s] [ 4%] 2024-08-08T21:00:14.8407524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0132s] [ 4%] 2024-08-08T21:00:14.8409119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0134s] [ 4%] 2024-08-08T21:00:14.8410723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 4%] 2024-08-08T21:00:14.8412324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 4%] 2024-08-08T21:00:14.8413930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 4%] 2024-08-08T21:00:14.8415525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 4%] 2024-08-08T21:00:14.8417118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0081s] [ 4%] 2024-08-08T21:00:14.8418739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 4%] 2024-08-08T21:00:14.8420324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 4%] 2024-08-08T21:00:14.8421987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 4%] 2024-08-08T21:00:14.8423628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 4%] 2024-08-08T21:00:14.8425234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 4%] 2024-08-08T21:00:14.8426817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 4%] 2024-08-08T21:00:14.8428474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 4%] 2024-08-08T21:00:14.8430111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0080s] [ 5%] 2024-08-08T21:00:14.8431729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 5%] 2024-08-08T21:00:14.8433317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 5%] 2024-08-08T21:00:14.8434924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 5%] 2024-08-08T21:00:14.8436525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 5%] 2024-08-08T21:00:14.8438120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 5%] 2024-08-08T21:00:14.8439730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 5%] 2024-08-08T21:00:14.8441335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 5%] 2024-08-08T21:00:14.8442944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0085s] [ 5%] 2024-08-08T21:00:14.8444590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 5%] 2024-08-08T21:00:14.8446253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 5%] 2024-08-08T21:00:14.8447851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 5%] 2024-08-08T21:00:14.8449443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 5%] 2024-08-08T21:00:14.8451068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 5%] 2024-08-08T21:00:14.8452690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 5%] 2024-08-08T21:00:14.8454289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 5%] 2024-08-08T21:00:14.8455870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0080s] [ 5%] 2024-08-08T21:00:14.8457537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 5%] 2024-08-08T21:00:14.8459529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 5%] 2024-08-08T21:00:14.8461174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 5%] 2024-08-08T21:00:14.8462760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 5%] 2024-08-08T21:00:14.8464375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 5%] 2024-08-08T21:00:14.8465951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 5%] 2024-08-08T21:00:14.8467550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 5%] 2024-08-08T21:00:14.8469223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 5%] 2024-08-08T21:00:14.8470911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 5%] 2024-08-08T21:00:14.8472516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 5%] 2024-08-08T21:00:14.8474122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 5%] 2024-08-08T21:00:14.8475784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 5%] 2024-08-08T21:00:14.8477435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 5%] 2024-08-08T21:00:14.8479025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 5%] 2024-08-08T21:00:14.8480614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 5%] 2024-08-08T21:00:14.8482229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0082s] [ 5%] 2024-08-08T21:00:14.8483824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 5%] 2024-08-08T21:00:14.8485431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 5%] 2024-08-08T21:00:14.8487030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 5%] 2024-08-08T21:00:14.8488619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 5%] 2024-08-08T21:00:14.8490221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 5%] 2024-08-08T21:00:14.8491840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 5%] 2024-08-08T21:00:14.8493488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 5%] 2024-08-08T21:00:14.8495066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0084s] [ 5%] 2024-08-08T21:00:14.8496671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 5%] 2024-08-08T21:00:14.8498300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 5%] 2024-08-08T21:00:14.8499967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 5%] 2024-08-08T21:00:14.8501533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 5%] 2024-08-08T21:00:14.8503137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 5%] 2024-08-08T21:00:14.8504712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 5%] 2024-08-08T21:00:14.8506295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 5%] 2024-08-08T21:00:14.8507886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 5%] 2024-08-08T21:00:14.8509473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 5%] 2024-08-08T21:00:14.8511066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 5%] 2024-08-08T21:00:14.8512644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 5%] 2024-08-08T21:00:14.8514243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0092s] [ 5%] 2024-08-08T21:00:14.8515894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0098s] [ 5%] 2024-08-08T21:00:14.8517559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 5%] 2024-08-08T21:00:14.8519171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 5%] 2024-08-08T21:00:14.8520792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0133s] [ 5%] 2024-08-08T21:00:14.8522449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0136s] [ 5%] 2024-08-08T21:00:14.8524095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0087s] [ 5%] 2024-08-08T21:00:14.8525717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 5%] 2024-08-08T21:00:14.8527319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0096s] [ 5%] 2024-08-08T21:00:14.8528944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 5%] 2024-08-08T21:00:14.8530538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 5%] 2024-08-08T21:00:14.8532162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 5%] 2024-08-08T21:00:14.8533759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0136s] [ 5%] 2024-08-08T21:00:14.8535399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0136s] [ 5%] 2024-08-08T21:00:14.8536996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0089s] [ 5%] 2024-08-08T21:00:14.8538665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 5%] 2024-08-08T21:00:14.8540315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0107s] [ 5%] 2024-08-08T21:00:14.8541931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0110s] [ 5%] 2024-08-08T21:00:14.8543519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 5%] 2024-08-08T21:00:14.8545169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 5%] 2024-08-08T21:00:14.8546836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0142s] [ 5%] 2024-08-08T21:00:14.8548438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0142s] [ 5%] 2024-08-08T21:00:14.8550053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0102s] [ 5%] 2024-08-08T21:00:14.8551666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 5%] 2024-08-08T21:00:14.8553273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 5%] 2024-08-08T21:00:14.8554858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0093s] [ 6%] 2024-08-08T21:00:14.8556521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 6%] 2024-08-08T21:00:14.8558131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 6%] 2024-08-08T21:00:14.8560171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0132s] [ 6%] 2024-08-08T21:00:14.8561794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0135s] [ 6%] 2024-08-08T21:00:14.8563490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0087s] [ 6%] 2024-08-08T21:00:14.8565189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 6%] 2024-08-08T21:00:14.8566774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0144s] [ 6%] 2024-08-08T21:00:14.8568447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0156s] [ 6%] 2024-08-08T21:00:14.8570118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0076s] [ 6%] 2024-08-08T21:00:14.8571735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 6%] 2024-08-08T21:00:14.8573327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0216s] [ 6%] 2024-08-08T21:00:14.8574958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0222s] [ 6%] 2024-08-08T21:00:14.8576564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0114s] [ 6%] 2024-08-08T21:00:14.8578187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0115s] [ 6%] 2024-08-08T21:00:14.8579771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0146s] [ 6%] 2024-08-08T21:00:14.8581382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0167s] [ 6%] 2024-08-08T21:00:14.8582991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 6%] 2024-08-08T21:00:14.8584586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 6%] 2024-08-08T21:00:14.8586248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0218s] [ 6%] 2024-08-08T21:00:14.8587908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0224s] [ 6%] 2024-08-08T21:00:14.8589519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0118s] [ 6%] 2024-08-08T21:00:14.8591122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0120s] [ 6%] 2024-08-08T21:00:14.8592770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0162s] [ 6%] 2024-08-08T21:00:14.8594425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0181s] [ 6%] 2024-08-08T21:00:14.8596028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0084s] [ 6%] 2024-08-08T21:00:14.8597630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0083s] [ 6%] 2024-08-08T21:00:14.8599228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0232s] [ 6%] 2024-08-08T21:00:14.8600844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0236s] [ 6%] 2024-08-08T21:00:14.8602434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0143s] [ 6%] 2024-08-08T21:00:14.8604060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0140s] [ 6%] 2024-08-08T21:00:14.8605650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0137s] [ 6%] 2024-08-08T21:00:14.8607262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0150s] [ 6%] 2024-08-08T21:00:14.8608889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 6%] 2024-08-08T21:00:14.8610559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 6%] 2024-08-08T21:00:14.8612180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0212s] [ 6%] 2024-08-08T21:00:14.8614578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0220s] [ 6%] 2024-08-08T21:00:14.8616289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0115s] [ 6%] 2024-08-08T21:00:14.8618171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0115s] [ 6%] 2024-08-08T21:00:14.8619900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0073s] [ 6%] 2024-08-08T21:00:14.8621420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 6%] 2024-08-08T21:00:14.8623028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 6%] 2024-08-08T21:00:14.8624634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 6%] 2024-08-08T21:00:14.8626242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0095s] [ 6%] 2024-08-08T21:00:14.8627845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 6%] 2024-08-08T21:00:14.8629462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 6%] 2024-08-08T21:00:14.8631062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 6%] 2024-08-08T21:00:14.8632659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0070s] [ 6%] 2024-08-08T21:00:14.8634315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 6%] 2024-08-08T21:00:14.8635973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 6%] 2024-08-08T21:00:14.8637598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 6%] 2024-08-08T21:00:14.8639157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0096s] [ 6%] 2024-08-08T21:00:14.8640822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 6%] 2024-08-08T21:00:14.8642499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0076s] [ 6%] 2024-08-08T21:00:14.8644090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 6%] 2024-08-08T21:00:14.8645682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 6%] 2024-08-08T21:00:14.8647297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 6%] 2024-08-08T21:00:14.8648884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 6%] 2024-08-08T21:00:14.8650491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 6%] 2024-08-08T21:00:14.8652081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0102s] [ 6%] 2024-08-08T21:00:14.8653709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 6%] 2024-08-08T21:00:14.8655374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0079s] [ 6%] 2024-08-08T21:00:14.8656997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 6%] 2024-08-08T21:00:14.8659108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 6%] 2024-08-08T21:00:14.8660710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 6%] 2024-08-08T21:00:14.8662294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 6%] 2024-08-08T21:00:14.8663997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 6%] 2024-08-08T21:00:14.8665725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0095s] [ 6%] 2024-08-08T21:00:14.8667248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 6%] 2024-08-08T21:00:14.8668910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 6%] 2024-08-08T21:00:14.8670462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 6%] 2024-08-08T21:00:14.8672071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 6%] 2024-08-08T21:00:14.8673712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 6%] 2024-08-08T21:00:14.8675235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 6%] 2024-08-08T21:00:14.8676845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 6%] 2024-08-08T21:00:14.8678420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0088s] [ 6%] 2024-08-08T21:00:14.8680027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 6%] 2024-08-08T21:00:14.8681676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 6%] 2024-08-08T21:00:14.8683367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 7%] 2024-08-08T21:00:14.8684940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0068s] [ 7%] 2024-08-08T21:00:14.8686542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 7%] 2024-08-08T21:00:14.8688159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 7%] 2024-08-08T21:00:14.8689807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 7%] 2024-08-08T21:00:14.8691382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0092s] [ 7%] 2024-08-08T21:00:14.8692976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 7%] 2024-08-08T21:00:14.8694591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 7%] 2024-08-08T21:00:14.8696187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 7%] 2024-08-08T21:00:14.8697775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 7%] 2024-08-08T21:00:14.8699362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 7%] 2024-08-08T21:00:14.8700963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 7%] 2024-08-08T21:00:14.8702536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 7%] 2024-08-08T21:00:14.8704230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 7%] 2024-08-08T21:00:14.8705869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 7%] 2024-08-08T21:00:14.8707478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 7%] 2024-08-08T21:00:14.8709076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 7%] 2024-08-08T21:00:14.8710691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 7%] 2024-08-08T21:00:14.8712344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0088s] [ 7%] 2024-08-08T21:00:14.8713910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0089s] [ 7%] 2024-08-08T21:00:14.8715498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0091s] [ 7%] 2024-08-08T21:00:14.8717073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0116s] [ 7%] 2024-08-08T21:00:14.8718686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 7%] 2024-08-08T21:00:14.8720259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0102s] [ 7%] 2024-08-08T21:00:14.8721862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0102s] [ 7%] 2024-08-08T21:00:14.8723454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0258s] [ 7%] 2024-08-08T21:00:14.8725068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0286s] [ 7%] 2024-08-08T21:00:14.8726654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0104s] [ 7%] 2024-08-08T21:00:14.8728304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0106s] [ 7%] 2024-08-08T21:00:14.8729971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0396s] [ 7%] 2024-08-08T21:00:14.8731581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0415s] [ 7%] 2024-08-08T21:00:14.8733198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0175s] [ 7%] 2024-08-08T21:00:14.8734865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0175s] [ 7%] 2024-08-08T21:00:14.8736531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0265s] [ 7%] 2024-08-08T21:00:14.8738129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0300s] [ 7%] 2024-08-08T21:00:14.8739738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0108s] [ 7%] 2024-08-08T21:00:14.8741350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0108s] [ 7%] 2024-08-08T21:00:14.8742963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0400s] [ 7%] 2024-08-08T21:00:14.8744566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0418s] [ 7%] 2024-08-08T21:00:14.8746181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0185s] [ 7%] 2024-08-08T21:00:14.8747795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0183s] [ 7%] 2024-08-08T21:00:14.8749379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0293s] [ 7%] 2024-08-08T21:00:14.8762016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0328s] [ 7%] 2024-08-08T21:00:14.8763864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0118s] [ 7%] 2024-08-08T21:00:14.8765506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0116s] [ 7%] 2024-08-08T21:00:14.8767118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0425s] [ 7%] 2024-08-08T21:00:14.8768832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0438s] [ 7%] 2024-08-08T21:00:14.8770507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0229s] [ 7%] 2024-08-08T21:00:14.8772136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0227s] [ 7%] 2024-08-08T21:00:14.8773726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0250s] [ 7%] 2024-08-08T21:00:14.8775353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0270s] [ 7%] 2024-08-08T21:00:14.8776950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0100s] [ 7%] 2024-08-08T21:00:14.8778600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 7%] 2024-08-08T21:00:14.8780198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0391s] [ 7%] 2024-08-08T21:00:14.8781816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0406s] [ 7%] 2024-08-08T21:00:14.8783422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0172s] [ 7%] 2024-08-08T21:00:14.8785018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0171s] [ 7%] 2024-08-08T21:00:14.8786668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0073s] [ 7%] 2024-08-08T21:00:14.8788311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 7%] 2024-08-08T21:00:14.8789912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 7%] 2024-08-08T21:00:14.8791548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 7%] 2024-08-08T21:00:14.8793210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0095s] [ 7%] 2024-08-08T21:00:14.8794808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0098s] [ 7%] 2024-08-08T21:00:14.8796407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 7%] 2024-08-08T21:00:14.8798013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 7%] 2024-08-08T21:00:14.8799619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0068s] [ 7%] 2024-08-08T21:00:14.8801216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 7%] 2024-08-08T21:00:14.8802798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 7%] 2024-08-08T21:00:14.8804412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 7%] 2024-08-08T21:00:14.8806002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 7%] 2024-08-08T21:00:14.8807617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 7%] 2024-08-08T21:00:14.8809250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0076s] [ 7%] 2024-08-08T21:00:14.8810920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 7%] 2024-08-08T21:00:14.8812494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0074s] [ 7%] 2024-08-08T21:00:14.8814103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 7%] 2024-08-08T21:00:14.8815730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 7%] 2024-08-08T21:00:14.8817398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 7%] 2024-08-08T21:00:14.8818985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0097s] [ 7%] 2024-08-08T21:00:14.8820582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 8%] 2024-08-08T21:00:14.8822191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 8%] 2024-08-08T21:00:14.8823787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 8%] 2024-08-08T21:00:14.8825374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0067s] [ 8%] 2024-08-08T21:00:14.8826965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 8%] 2024-08-08T21:00:14.8828559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 8%] 2024-08-08T21:00:14.8830136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 8%] 2024-08-08T21:00:14.8831780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0095s] [ 8%] 2024-08-08T21:00:14.8833439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0093s] [ 8%] 2024-08-08T21:00:14.8835045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 8%] 2024-08-08T21:00:14.8836643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 8%] 2024-08-08T21:00:14.8838262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 8%] 2024-08-08T21:00:14.8839920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 8%] 2024-08-08T21:00:14.8841489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 8%] 2024-08-08T21:00:14.8843084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 8%] 2024-08-08T21:00:14.8844671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 8%] 2024-08-08T21:00:14.8846283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0088s] [ 8%] 2024-08-08T21:00:14.8847870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 8%] 2024-08-08T21:00:14.8849485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 8%] 2024-08-08T21:00:14.8851066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0067s] [ 8%] 2024-08-08T21:00:14.8852675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 8%] 2024-08-08T21:00:14.8854242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 8%] 2024-08-08T21:00:14.8855871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 8%] 2024-08-08T21:00:14.8857585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0090s] [ 8%] 2024-08-08T21:00:14.8859601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 8%] 2024-08-08T21:00:14.8861286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 8%] 2024-08-08T21:00:14.8862988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 8%] 2024-08-08T21:00:14.8864660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 8%] 2024-08-08T21:00:14.8866246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 8%] 2024-08-08T21:00:14.8867839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 8%] 2024-08-08T21:00:14.8869432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 8%] 2024-08-08T21:00:14.8871029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0096s] [ 8%] 2024-08-08T21:00:14.8872617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 8%] 2024-08-08T21:00:14.8874206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 8%] 2024-08-08T21:00:14.8875829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 8%] 2024-08-08T21:00:14.8877400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 8%] 2024-08-08T21:00:14.8879056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 8%] 2024-08-08T21:00:14.8880691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 8%] 2024-08-08T21:00:14.8882295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 8%] 2024-08-08T21:00:14.8883872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0086s] [ 8%] 2024-08-08T21:00:14.8885523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0088s] [ 8%] 2024-08-08T21:00:14.8887145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 8%] 2024-08-08T21:00:14.8888747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 8%] 2024-08-08T21:00:14.8890312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0098s] [ 8%] 2024-08-08T21:00:14.8891902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 8%] 2024-08-08T21:00:14.8893496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0092s] [ 8%] 2024-08-08T21:00:14.8895073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 8%] 2024-08-08T21:00:14.8896675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 8%] 2024-08-08T21:00:14.8898275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 8%] 2024-08-08T21:00:14.8899882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 8%] 2024-08-08T21:00:14.8901475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 8%] 2024-08-08T21:00:14.8903119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 8%] 2024-08-08T21:00:14.8904752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 8%] 2024-08-08T21:00:14.8906343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 8%] 2024-08-08T21:00:14.8907919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 8%] 2024-08-08T21:00:14.8909547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 8%] 2024-08-08T21:00:14.8911207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 8%] 2024-08-08T21:00:14.8912796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 8%] 2024-08-08T21:00:14.8914417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 8%] 2024-08-08T21:00:14.8915997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 8%] 2024-08-08T21:00:14.8917600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 8%] 2024-08-08T21:00:14.8919171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 8%] 2024-08-08T21:00:14.8920779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 8%] 2024-08-08T21:00:14.8922366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 8%] 2024-08-08T21:00:14.8923980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 8%] 2024-08-08T21:00:14.8925561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 8%] 2024-08-08T21:00:14.8927217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 8%] 2024-08-08T21:00:14.8928862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 8%] 2024-08-08T21:00:14.8930447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 8%] 2024-08-08T21:00:14.8932080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 8%] 2024-08-08T21:00:14.8933707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 8%] 2024-08-08T21:00:14.8935298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 8%] 2024-08-08T21:00:14.8936973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 8%] 2024-08-08T21:00:14.8938479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 8%] 2024-08-08T21:00:14.8940068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 8%] 2024-08-08T21:00:14.8941656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 8%] 2024-08-08T21:00:14.8943241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 8%] 2024-08-08T21:00:14.8944819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 8%] 2024-08-08T21:00:14.8946417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 9%] 2024-08-08T21:00:14.8948005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 9%] 2024-08-08T21:00:14.8949660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 9%] 2024-08-08T21:00:14.8951303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 9%] 2024-08-08T21:00:14.8952922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 9%] 2024-08-08T21:00:14.8954498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 9%] 2024-08-08T21:00:14.8956201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 9%] 2024-08-08T21:00:14.8957828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 9%] 2024-08-08T21:00:14.8959835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 9%] 2024-08-08T21:00:14.8961450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 9%] 2024-08-08T21:00:14.8963069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 9%] 2024-08-08T21:00:14.8964659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 9%] 2024-08-08T21:00:14.8966257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 9%] 2024-08-08T21:00:14.8967846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0073s] [ 9%] 2024-08-08T21:00:14.8969443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 9%] 2024-08-08T21:00:14.8971029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 9%] 2024-08-08T21:00:14.8972605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 9%] 2024-08-08T21:00:14.8974293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 9%] 2024-08-08T21:00:14.8975962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 9%] 2024-08-08T21:00:14.8977570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 9%] 2024-08-08T21:00:14.8979221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 9%] 2024-08-08T21:00:14.8980877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 9%] 2024-08-08T21:00:14.8982457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 9%] 2024-08-08T21:00:14.8984016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 9%] 2024-08-08T21:00:14.8985614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 9%] 2024-08-08T21:00:14.8987196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 9%] 2024-08-08T21:00:14.8988795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 9%] 2024-08-08T21:00:14.8990361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 9%] 2024-08-08T21:00:14.8991963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 9%] 2024-08-08T21:00:14.8993538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 9%] 2024-08-08T21:00:14.8995134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 9%] 2024-08-08T21:00:14.8996736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 9%] 2024-08-08T21:00:14.8998380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 9%] 2024-08-08T21:00:14.8999949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 9%] 2024-08-08T21:00:14.9001526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 9%] 2024-08-08T21:00:14.9003153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 9%] 2024-08-08T21:00:14.9004798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 9%] 2024-08-08T21:00:14.9006376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 9%] 2024-08-08T21:00:14.9007951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 9%] 2024-08-08T21:00:14.9009541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 9%] 2024-08-08T21:00:14.9011115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 9%] 2024-08-08T21:00:14.9012698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 9%] 2024-08-08T21:00:14.9014282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 9%] 2024-08-08T21:00:14.9015878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 9%] 2024-08-08T21:00:14.9017456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 9%] 2024-08-08T21:00:14.9019017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 9%] 2024-08-08T21:00:14.9020653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 9%] 2024-08-08T21:00:14.9022268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 9%] 2024-08-08T21:00:14.9023854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 9%] 2024-08-08T21:00:14.9025421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 9%] 2024-08-08T21:00:14.9027063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 9%] 2024-08-08T21:00:14.9028673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 9%] 2024-08-08T21:00:14.9030263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 9%] 2024-08-08T21:00:14.9031825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 9%] 2024-08-08T21:00:14.9033433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 9%] 2024-08-08T21:00:14.9034999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 9%] 2024-08-08T21:00:14.9036568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 9%] 2024-08-08T21:00:14.9038158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 9%] 2024-08-08T21:00:14.9039739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 9%] 2024-08-08T21:00:14.9041321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 9%] 2024-08-08T21:00:14.9042940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 9%] 2024-08-08T21:00:14.9044576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 9%] 2024-08-08T21:00:14.9046154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 9%] 2024-08-08T21:00:14.9047735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 9%] 2024-08-08T21:00:14.9049348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 9%] 2024-08-08T21:00:14.9050989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 9%] 2024-08-08T21:00:14.9052562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 9%] 2024-08-08T21:00:14.9054119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 9%] 2024-08-08T21:00:14.9055719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 9%] 2024-08-08T21:00:14.9057331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 9%] 2024-08-08T21:00:14.9059321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 9%] 2024-08-08T21:00:14.9060931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 9%] 2024-08-08T21:00:14.9062545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 9%] 2024-08-08T21:00:14.9064109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 9%] 2024-08-08T21:00:14.9065706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 9%] 2024-08-08T21:00:14.9067354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 9%] 2024-08-08T21:00:14.9069024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 9%] 2024-08-08T21:00:14.9070585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 9%] 2024-08-08T21:00:14.9072168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 10%] 2024-08-08T21:00:14.9073817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 10%] 2024-08-08T21:00:14.9075524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 10%] 2024-08-08T21:00:14.9077110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 10%] 2024-08-08T21:00:14.9078696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 10%] 2024-08-08T21:00:14.9080289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 10%] 2024-08-08T21:00:14.9081865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 10%] 2024-08-08T21:00:14.9083438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 10%] 2024-08-08T21:00:14.9085014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 10%] 2024-08-08T21:00:14.9086617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 10%] 2024-08-08T21:00:14.9088182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 10%] 2024-08-08T21:00:14.9089790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 10%] 2024-08-08T21:00:14.9091437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 10%] 2024-08-08T21:00:14.9093012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 10%] 2024-08-08T21:00:14.9094604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 10%] 2024-08-08T21:00:14.9096251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0086s] [ 10%] 2024-08-08T21:00:14.9097908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0085s] [ 10%] 2024-08-08T21:00:14.9099484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 10%] 2024-08-08T21:00:14.9101080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 10%] 2024-08-08T21:00:14.9102663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 10%] 2024-08-08T21:00:14.9104287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 10%] 2024-08-08T21:00:14.9105878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 10%] 2024-08-08T21:00:14.9107484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 10%] 2024-08-08T21:00:14.9109092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 10%] 2024-08-08T21:00:14.9110679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 10%] 2024-08-08T21:00:14.9112261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 10%] 2024-08-08T21:00:14.9113892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 10%] 2024-08-08T21:00:14.9115546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0102s] [ 10%] 2024-08-08T21:00:14.9117140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 10%] 2024-08-08T21:00:14.9118748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 10%] 2024-08-08T21:00:14.9120387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 10%] 2024-08-08T21:00:14.9122043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0106s] [ 10%] 2024-08-08T21:00:14.9123632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0109s] [ 10%] 2024-08-08T21:00:14.9125208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 10%] 2024-08-08T21:00:14.9126820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 10%] 2024-08-08T21:00:14.9128398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0116s] [ 10%] 2024-08-08T21:00:14.9130008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0116s] [ 10%] 2024-08-08T21:00:14.9131610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 10%] 2024-08-08T21:00:14.9133234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 10%] 2024-08-08T21:00:14.9134806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0081s] [ 10%] 2024-08-08T21:00:14.9136445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 10%] 2024-08-08T21:00:14.9138061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 10%] 2024-08-08T21:00:14.9139659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 10%] 2024-08-08T21:00:14.9141228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0090s] [ 10%] 2024-08-08T21:00:14.9142856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0090s] [ 10%] 2024-08-08T21:00:14.9144495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 10%] 2024-08-08T21:00:14.9146072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 10%] 2024-08-08T21:00:14.9147648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 10%] 2024-08-08T21:00:14.9149234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 10%] 2024-08-08T21:00:14.9150823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 10%] 2024-08-08T21:00:14.9152403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 10%] 2024-08-08T21:00:14.9153995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 10%] 2024-08-08T21:00:14.9155591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 10%] 2024-08-08T21:00:14.9157230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 10%] 2024-08-08T21:00:14.9159599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 10%] 2024-08-08T21:00:14.9161499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 10%] 2024-08-08T21:00:14.9163203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 10%] 2024-08-08T21:00:14.9164765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 10%] 2024-08-08T21:00:14.9166352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 10%] 2024-08-08T21:00:14.9167983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 10%] 2024-08-08T21:00:14.9169664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 10%] 2024-08-08T21:00:14.9171230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 10%] 2024-08-08T21:00:14.9172832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 10%] 2024-08-08T21:00:14.9174405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 10%] 2024-08-08T21:00:14.9176004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 10%] 2024-08-08T21:00:14.9177561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 10%] 2024-08-08T21:00:14.9179135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 10%] 2024-08-08T21:00:14.9180729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 10%] 2024-08-08T21:00:14.9182311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 10%] 2024-08-08T21:00:14.9183892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 10%] 2024-08-08T21:00:14.9185515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 10%] 2024-08-08T21:00:14.9187150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 10%] 2024-08-08T21:00:14.9188728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 10%] 2024-08-08T21:00:14.9190369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 10%] 2024-08-08T21:00:14.9191993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 10%] 2024-08-08T21:00:14.9193582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 10%] 2024-08-08T21:00:14.9195163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 10%] 2024-08-08T21:00:14.9196759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 10%] 2024-08-08T21:00:14.9198337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 11%] 2024-08-08T21:00:14.9199904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 11%] 2024-08-08T21:00:14.9201507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 11%] 2024-08-08T21:00:14.9203086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 11%] 2024-08-08T21:00:14.9204687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 11%] 2024-08-08T21:00:14.9206260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 11%] 2024-08-08T21:00:14.9207912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 11%] 2024-08-08T21:00:14.9209588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 11%] 2024-08-08T21:00:14.9211181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 11%] 2024-08-08T21:00:14.9212739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 11%] 2024-08-08T21:00:14.9214362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 11%] 2024-08-08T21:00:14.9216000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 11%] 2024-08-08T21:00:14.9217568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 11%] 2024-08-08T21:00:14.9219158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 11%] 2024-08-08T21:00:14.9220748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 11%] 2024-08-08T21:00:14.9222336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 11%] 2024-08-08T21:00:14.9223910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 11%] 2024-08-08T21:00:14.9225496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 11%] 2024-08-08T21:00:14.9227080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 11%] 2024-08-08T21:00:14.9228662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 11%] 2024-08-08T21:00:14.9230231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 11%] 2024-08-08T21:00:14.9231846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 11%] 2024-08-08T21:00:14.9233495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 11%] 2024-08-08T21:00:14.9235055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0077s] [ 11%] 2024-08-08T21:00:14.9236643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 11%] 2024-08-08T21:00:14.9238249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 11%] 2024-08-08T21:00:14.9239904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 11%] 2024-08-08T21:00:14.9241462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 11%] 2024-08-08T21:00:14.9243066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 11%] 2024-08-08T21:00:14.9244632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 11%] 2024-08-08T21:00:14.9246231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 11%] 2024-08-08T21:00:14.9247801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 11%] 2024-08-08T21:00:14.9249377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 11%] 2024-08-08T21:00:14.9250994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0140s] [ 11%] 2024-08-08T21:00:14.9252645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0149s] [ 11%] 2024-08-08T21:00:14.9254310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0086s] [ 11%] 2024-08-08T21:00:14.9255966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 11%] 2024-08-08T21:00:14.9257584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0214s] [ 11%] 2024-08-08T21:00:14.9259655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0215s] [ 11%] 2024-08-08T21:00:14.9261404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0120s] [ 11%] 2024-08-08T21:00:14.9263090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0120s] [ 11%] 2024-08-08T21:00:14.9264700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0141s] [ 11%] 2024-08-08T21:00:14.9266291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0155s] [ 11%] 2024-08-08T21:00:14.9267890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0085s] [ 11%] 2024-08-08T21:00:14.9269523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 11%] 2024-08-08T21:00:14.9271112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0216s] [ 11%] 2024-08-08T21:00:14.9272739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0224s] [ 11%] 2024-08-08T21:00:14.9274346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0125s] [ 11%] 2024-08-08T21:00:14.9275969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0123s] [ 11%] 2024-08-08T21:00:14.9277553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0158s] [ 11%] 2024-08-08T21:00:14.9279230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0174s] [ 11%] 2024-08-08T21:00:14.9280886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0089s] [ 11%] 2024-08-08T21:00:14.9282510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 11%] 2024-08-08T21:00:14.9284105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0233s] [ 11%] 2024-08-08T21:00:14.9285750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0231s] [ 11%] 2024-08-08T21:00:14.9287414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0146s] [ 11%] 2024-08-08T21:00:14.9289015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0147s] [ 11%] 2024-08-08T21:00:14.9290616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0133s] [ 11%] 2024-08-08T21:00:14.9292220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0139s] [ 11%] 2024-08-08T21:00:14.9293827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 11%] 2024-08-08T21:00:14.9295427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 11%] 2024-08-08T21:00:14.9297040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0212s] [ 11%] 2024-08-08T21:00:14.9298648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0214s] [ 11%] 2024-08-08T21:00:14.9300259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0122s] [ 11%] 2024-08-08T21:00:14.9301905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0123s] [ 11%] 2024-08-08T21:00:14.9303566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0240s] [ 11%] 2024-08-08T21:00:14.9305159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0265s] [ 11%] 2024-08-08T21:00:14.9306750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0106s] [ 11%] 2024-08-08T21:00:14.9308420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0106s] [ 11%] 2024-08-08T21:00:14.9310067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0383s] [ 11%] 2024-08-08T21:00:14.9311685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0392s] [ 11%] 2024-08-08T21:00:14.9313283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0177s] [ 11%] 2024-08-08T21:00:14.9314918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0181s] [ 11%] 2024-08-08T21:00:14.9316507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0245s] [ 11%] 2024-08-08T21:00:14.9318124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0282s] [ 11%] 2024-08-08T21:00:14.9319721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0110s] [ 11%] 2024-08-08T21:00:14.9321352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0108s] [ 11%] 2024-08-08T21:00:14.9322944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0388s] [ 12%] 2024-08-08T21:00:14.9324555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0399s] [ 12%] 2024-08-08T21:00:14.9326220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0183s] [ 12%] 2024-08-08T21:00:14.9327884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0185s] [ 12%] 2024-08-08T21:00:14.9329485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0275s] [ 12%] 2024-08-08T21:00:14.9331146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0314s] [ 12%] 2024-08-08T21:00:14.9332816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0115s] [ 12%] 2024-08-08T21:00:14.9334411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0119s] [ 12%] 2024-08-08T21:00:14.9336020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0413s] [ 12%] 2024-08-08T21:00:14.9337631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0420s] [ 12%] 2024-08-08T21:00:14.9339252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0231s] [ 12%] 2024-08-08T21:00:14.9340854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0227s] [ 12%] 2024-08-08T21:00:14.9342436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0229s] [ 12%] 2024-08-08T21:00:14.9344058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0247s] [ 12%] 2024-08-08T21:00:14.9345653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0103s] [ 12%] 2024-08-08T21:00:14.9347262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 12%] 2024-08-08T21:00:14.9348896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0378s] [ 12%] 2024-08-08T21:00:14.9350568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0388s] [ 12%] 2024-08-08T21:00:14.9352204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0176s] [ 12%] 2024-08-08T21:00:14.9353828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0177s] [ 12%] 2024-08-08T21:00:14.9355466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0092s] [ 12%] 2024-08-08T21:00:14.9357134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 12%] 2024-08-08T21:00:14.9359116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0079s] [ 12%] 2024-08-08T21:00:14.9360752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 12%] 2024-08-08T21:00:14.9362370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0138s] [ 12%] 2024-08-08T21:00:14.9363968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0141s] [ 12%] 2024-08-08T21:00:14.9365572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 12%] 2024-08-08T21:00:14.9367176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0098s] [ 12%] 2024-08-08T21:00:14.9368783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0094s] [ 12%] 2024-08-08T21:00:14.9370370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 12%] 2024-08-08T21:00:14.9372055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 12%] 2024-08-08T21:00:14.9373725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 12%] 2024-08-08T21:00:14.9375330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0141s] [ 12%] 2024-08-08T21:00:14.9376920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0142s] [ 12%] 2024-08-08T21:00:14.9378569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0098s] [ 12%] 2024-08-08T21:00:14.9380251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0099s] [ 12%] 2024-08-08T21:00:14.9381844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0105s] [ 12%] 2024-08-08T21:00:14.9383443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0108s] [ 12%] 2024-08-08T21:00:14.9385030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0078s] [ 12%] 2024-08-08T21:00:14.9386649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 12%] 2024-08-08T21:00:14.9388232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0153s] [ 12%] 2024-08-08T21:00:14.9389852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0154s] [ 12%] 2024-08-08T21:00:14.9391447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0104s] [ 12%] 2024-08-08T21:00:14.9393068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0104s] [ 12%] 2024-08-08T21:00:14.9394643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0092s] [ 12%] 2024-08-08T21:00:14.9396291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 12%] 2024-08-08T21:00:14.9397915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0076s] [ 12%] 2024-08-08T21:00:14.9399495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 12%] 2024-08-08T21:00:14.9401088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0134s] [ 12%] 2024-08-08T21:00:14.9402721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0141s] [ 12%] 2024-08-08T21:00:14.9404383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0097s] [ 12%] 2024-08-08T21:00:14.9405968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 12%] 2024-08-08T21:00:14.9407565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0086s] [ 12%] 2024-08-08T21:00:14.9409157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0085s] [ 12%] 2024-08-08T21:00:14.9410729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 12%] 2024-08-08T21:00:14.9412313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 12%] 2024-08-08T21:00:14.9413912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0124s] [ 12%] 2024-08-08T21:00:14.9415512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0122s] [ 12%] 2024-08-08T21:00:14.9417097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 12%] 2024-08-08T21:00:14.9418748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 12%] 2024-08-08T21:00:14.9420394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0089s] [ 12%] 2024-08-08T21:00:14.9422000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 12%] 2024-08-08T21:00:14.9423564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 12%] 2024-08-08T21:00:14.9425203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 12%] 2024-08-08T21:00:14.9426836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0127s] [ 12%] 2024-08-08T21:00:14.9428436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0125s] [ 12%] 2024-08-08T21:00:14.9430018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 12%] 2024-08-08T21:00:14.9431636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 12%] 2024-08-08T21:00:14.9433221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0100s] [ 12%] 2024-08-08T21:00:14.9434804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0099s] [ 12%] 2024-08-08T21:00:14.9436394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 12%] 2024-08-08T21:00:14.9437977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 12%] 2024-08-08T21:00:14.9439576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0138s] [ 12%] 2024-08-08T21:00:14.9441159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0139s] [ 12%] 2024-08-08T21:00:14.9442809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0089s] [ 12%] 2024-08-08T21:00:14.9444465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 12%] 2024-08-08T21:00:14.9446052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0083s] [ 12%] 2024-08-08T21:00:14.9447629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 12%] 2024-08-08T21:00:14.9449253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 13%] 2024-08-08T21:00:14.9450880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 13%] 2024-08-08T21:00:14.9452494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0121s] [ 13%] 2024-08-08T21:00:14.9454103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0123s] [ 13%] 2024-08-08T21:00:14.9455679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0083s] [ 13%] 2024-08-08T21:00:14.9457285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0083s] [ 13%] 2024-08-08T21:00:14.9459276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0456s] [ 13%] 2024-08-08T21:00:14.9460934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0517s] [ 13%] 2024-08-08T21:00:14.9462539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0144s] [ 13%] 2024-08-08T21:00:14.9464158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0145s] [ 13%] 2024-08-08T21:00:14.9465840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0731s] [ 13%] 2024-08-08T21:00:14.9467547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0770s] [ 13%] 2024-08-08T21:00:14.9469151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0290s] [ 13%] 2024-08-08T21:00:14.9470763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0289s] [ 13%] 2024-08-08T21:00:14.9472448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0470s] [ 13%] 2024-08-08T21:00:14.9474124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0539s] [ 13%] 2024-08-08T21:00:14.9475733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0156s] [ 13%] 2024-08-08T21:00:14.9477328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0161s] [ 13%] 2024-08-08T21:00:14.9478943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0736s] [ 13%] 2024-08-08T21:00:14.9480662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0768s] [ 13%] 2024-08-08T21:00:14.9482174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0301s] [ 13%] 2024-08-08T21:00:14.9483785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0303s] [ 13%] 2024-08-08T21:00:14.9485408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0525s] [ 13%] 2024-08-08T21:00:14.9487007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0597s] [ 13%] 2024-08-08T21:00:14.9488617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0169s] [ 13%] 2024-08-08T21:00:14.9490265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0166s] [ 13%] 2024-08-08T21:00:14.9492024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0781s] [ 13%] 2024-08-08T21:00:14.9493532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0806s] [ 13%] 2024-08-08T21:00:14.9495136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0392s] [ 13%] 2024-08-08T21:00:14.9496813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0393s] [ 13%] 2024-08-08T21:00:14.9498443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0441s] [ 13%] 2024-08-08T21:00:14.9500057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0486s] [ 13%] 2024-08-08T21:00:14.9501649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0140s] [ 13%] 2024-08-08T21:00:14.9503275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0143s] [ 13%] 2024-08-08T21:00:14.9504867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0723s] [ 13%] 2024-08-08T21:00:14.9506471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0755s] [ 13%] 2024-08-08T21:00:14.9508090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0285s] [ 13%] 2024-08-08T21:00:14.9509697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0283s] [ 13%] 2024-08-08T21:00:14.9511292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0093s] [ 13%] 2024-08-08T21:00:14.9512945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 13%] 2024-08-08T21:00:14.9514605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0078s] [ 13%] 2024-08-08T21:00:14.9516199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 13%] 2024-08-08T21:00:14.9517802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0139s] [ 13%] 2024-08-08T21:00:14.9519454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0134s] [ 13%] 2024-08-08T21:00:14.9521117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 13%] 2024-08-08T21:00:14.9522733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0096s] [ 13%] 2024-08-08T21:00:14.9524322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0092s] [ 13%] 2024-08-08T21:00:14.9525926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 13%] 2024-08-08T21:00:14.9527539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 13%] 2024-08-08T21:00:14.9529127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 13%] 2024-08-08T21:00:14.9530719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0137s] [ 13%] 2024-08-08T21:00:14.9532349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0136s] [ 13%] 2024-08-08T21:00:14.9533940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0097s] [ 13%] 2024-08-08T21:00:14.9535560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0097s] [ 13%] 2024-08-08T21:00:14.9537189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0103s] [ 13%] 2024-08-08T21:00:14.9538849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0108s] [ 13%] 2024-08-08T21:00:14.9540435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 13%] 2024-08-08T21:00:14.9542050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 13%] 2024-08-08T21:00:14.9543694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0144s] [ 13%] 2024-08-08T21:00:14.9545343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0143s] [ 13%] 2024-08-08T21:00:14.9546949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0110s] [ 13%] 2024-08-08T21:00:14.9548554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0108s] [ 13%] 2024-08-08T21:00:14.9550157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 13%] 2024-08-08T21:00:14.9551804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 13%] 2024-08-08T21:00:14.9553398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 13%] 2024-08-08T21:00:14.9554968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 13%] 2024-08-08T21:00:14.9556570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0134s] [ 13%] 2024-08-08T21:00:14.9558183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0134s] [ 13%] 2024-08-08T21:00:14.9560274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 13%] 2024-08-08T21:00:14.9561965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0096s] [ 13%] 2024-08-08T21:00:14.9563538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0086s] [ 13%] 2024-08-08T21:00:14.9565141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 13%] 2024-08-08T21:00:14.9566769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0071s] [ 13%] 2024-08-08T21:00:14.9568438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 13%] 2024-08-08T21:00:14.9570023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0129s] [ 13%] 2024-08-08T21:00:14.9571629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0127s] [ 13%] 2024-08-08T21:00:14.9573219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 13%] 2024-08-08T21:00:14.9574839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0083s] [ 13%] 2024-08-08T21:00:14.9576410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 14%] 2024-08-08T21:00:14.9578016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 14%] 2024-08-08T21:00:14.9579592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 14%] 2024-08-08T21:00:14.9581172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 14%] 2024-08-08T21:00:14.9582766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0130s] [ 14%] 2024-08-08T21:00:14.9584440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0132s] [ 14%] 2024-08-08T21:00:14.9586094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0086s] [ 14%] 2024-08-08T21:00:14.9587688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 14%] 2024-08-08T21:00:14.9589282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0100s] [ 14%] 2024-08-08T21:00:14.9590911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0104s] [ 14%] 2024-08-08T21:00:14.9592567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0076s] [ 14%] 2024-08-08T21:00:14.9594147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 14%] 2024-08-08T21:00:14.9595746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0143s] [ 14%] 2024-08-08T21:00:14.9597342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0142s] [ 14%] 2024-08-08T21:00:14.9598926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0091s] [ 14%] 2024-08-08T21:00:14.9600538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 14%] 2024-08-08T21:00:14.9602114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0081s] [ 14%] 2024-08-08T21:00:14.9603710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 14%] 2024-08-08T21:00:14.9605270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 14%] 2024-08-08T21:00:14.9606909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 14%] 2024-08-08T21:00:14.9608536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0123s] [ 14%] 2024-08-08T21:00:14.9610133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0124s] [ 14%] 2024-08-08T21:00:14.9611693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0082s] [ 14%] 2024-08-08T21:00:14.9613329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 14%] 2024-08-08T21:00:14.9614965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 14%] 2024-08-08T21:00:14.9616555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 14%] 2024-08-08T21:00:14.9618163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 14%] 2024-08-08T21:00:14.9619776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 14%] 2024-08-08T21:00:14.9621477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 14%] 2024-08-08T21:00:14.9622977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 14%] 2024-08-08T21:00:14.9624591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 14%] 2024-08-08T21:00:14.9626201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 14%] 2024-08-08T21:00:14.9627801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 14%] 2024-08-08T21:00:14.9629394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 14%] 2024-08-08T21:00:14.9631038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 14%] 2024-08-08T21:00:14.9632684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 14%] 2024-08-08T21:00:14.9634265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 14%] 2024-08-08T21:00:14.9635922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 14%] 2024-08-08T21:00:14.9637561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 14%] 2024-08-08T21:00:14.9639197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 14%] 2024-08-08T21:00:14.9640753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 14%] 2024-08-08T21:00:14.9642371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 14%] 2024-08-08T21:00:14.9643965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 14%] 2024-08-08T21:00:14.9645575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 14%] 2024-08-08T21:00:14.9647157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 14%] 2024-08-08T21:00:14.9648780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 14%] 2024-08-08T21:00:14.9650372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 14%] 2024-08-08T21:00:14.9652024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 14%] 2024-08-08T21:00:14.9653645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 14%] 2024-08-08T21:00:14.9655284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 14%] 2024-08-08T21:00:14.9656875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 14%] 2024-08-08T21:00:14.9658770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 14%] 2024-08-08T21:00:14.9660509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 14%] 2024-08-08T21:00:14.9662177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 14%] 2024-08-08T21:00:14.9663782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 14%] 2024-08-08T21:00:14.9665380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 14%] 2024-08-08T21:00:14.9666990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0072s] [ 14%] 2024-08-08T21:00:14.9668586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 14%] 2024-08-08T21:00:14.9670195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 14%] 2024-08-08T21:00:14.9671795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 14%] 2024-08-08T21:00:14.9673393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 14%] 2024-08-08T21:00:14.9675012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 14%] 2024-08-08T21:00:14.9676602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 14%] 2024-08-08T21:00:14.9678285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 14%] 2024-08-08T21:00:14.9679937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0074s] [ 14%] 2024-08-08T21:00:14.9681555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 14%] 2024-08-08T21:00:14.9683203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 14%] 2024-08-08T21:00:14.9684870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 14%] 2024-08-08T21:00:14.9686458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0096s] [ 14%] 2024-08-08T21:00:14.9688074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 14%] 2024-08-08T21:00:14.9689671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 14%] 2024-08-08T21:00:14.9691286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 14%] 2024-08-08T21:00:14.9692889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0081s] [ 14%] 2024-08-08T21:00:14.9694488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0085s] [ 14%] 2024-08-08T21:00:14.9696102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 14%] 2024-08-08T21:00:14.9697705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 14%] 2024-08-08T21:00:14.9699328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0100s] [ 14%] 2024-08-08T21:00:14.9700958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 14%] 2024-08-08T21:00:14.9702621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0078s] [ 15%] 2024-08-08T21:00:14.9704218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 15%] 2024-08-08T21:00:14.9705809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0071s] [ 15%] 2024-08-08T21:00:14.9707447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 15%] 2024-08-08T21:00:14.9709079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 15%] 2024-08-08T21:00:14.9710679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 15%] 2024-08-08T21:00:14.9712268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0093s] [ 15%] 2024-08-08T21:00:14.9713884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 15%] 2024-08-08T21:00:14.9715464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 15%] 2024-08-08T21:00:14.9717074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 15%] 2024-08-08T21:00:14.9718652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 15%] 2024-08-08T21:00:14.9720270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 15%] 2024-08-08T21:00:14.9721842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 15%] 2024-08-08T21:00:14.9723489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 15%] 2024-08-08T21:00:14.9725117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 15%] 2024-08-08T21:00:14.9726722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 15%] 2024-08-08T21:00:14.9728327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 15%] 2024-08-08T21:00:14.9729970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 15%] 2024-08-08T21:00:14.9731613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 15%] 2024-08-08T21:00:14.9733200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 15%] 2024-08-08T21:00:14.9734784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 15%] 2024-08-08T21:00:14.9736376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 15%] 2024-08-08T21:00:14.9737983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 15%] 2024-08-08T21:00:14.9739575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 15%] 2024-08-08T21:00:14.9741195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 15%] 2024-08-08T21:00:14.9742781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 15%] 2024-08-08T21:00:14.9744360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 15%] 2024-08-08T21:00:14.9745960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 15%] 2024-08-08T21:00:14.9747574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 15%] 2024-08-08T21:00:14.9749220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 15%] 2024-08-08T21:00:14.9750800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 15%] 2024-08-08T21:00:14.9752456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 15%] 2024-08-08T21:00:14.9754088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 15%] 2024-08-08T21:00:14.9755760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 15%] 2024-08-08T21:00:14.9757335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 15%] 2024-08-08T21:00:14.9759351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 15%] 2024-08-08T21:00:14.9760958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 15%] 2024-08-08T21:00:14.9762533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 15%] 2024-08-08T21:00:14.9764120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 15%] 2024-08-08T21:00:14.9765710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 15%] 2024-08-08T21:00:14.9767298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 15%] 2024-08-08T21:00:14.9768905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 15%] 2024-08-08T21:00:14.9770467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 15%] 2024-08-08T21:00:14.9772137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 15%] 2024-08-08T21:00:14.9773814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 15%] 2024-08-08T21:00:14.9775388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 15%] 2024-08-08T21:00:14.9777033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 15%] 2024-08-08T21:00:14.9778685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 15%] 2024-08-08T21:00:14.9780249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 15%] 2024-08-08T21:00:14.9781843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 15%] 2024-08-08T21:00:14.9783411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 15%] 2024-08-08T21:00:14.9785005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 15%] 2024-08-08T21:00:14.9786561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 15%] 2024-08-08T21:00:14.9788147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 15%] 2024-08-08T21:00:14.9789723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 15%] 2024-08-08T21:00:14.9791325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 15%] 2024-08-08T21:00:14.9792887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 15%] 2024-08-08T21:00:14.9794529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 15%] 2024-08-08T21:00:14.9796151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 15%] 2024-08-08T21:00:14.9797723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 15%] 2024-08-08T21:00:14.9799293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 15%] 2024-08-08T21:00:14.9800973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 15%] 2024-08-08T21:00:14.9802631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 15%] 2024-08-08T21:00:14.9804213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 15%] 2024-08-08T21:00:14.9805796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 15%] 2024-08-08T21:00:14.9807384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 15%] 2024-08-08T21:00:14.9808965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 15%] 2024-08-08T21:00:14.9810536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 15%] 2024-08-08T21:00:14.9812123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 15%] 2024-08-08T21:00:14.9813704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 15%] 2024-08-08T21:00:14.9815268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 15%] 2024-08-08T21:00:14.9816860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 15%] 2024-08-08T21:00:14.9818466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 15%] 2024-08-08T21:00:14.9820108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 15%] 2024-08-08T21:00:14.9821685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0108s] [ 15%] 2024-08-08T21:00:14.9823299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0115s] [ 15%] 2024-08-08T21:00:14.9824939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 15%] 2024-08-08T21:00:14.9826601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 15%] 2024-08-08T21:00:14.9828192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0147s] [ 16%] 2024-08-08T21:00:14.9829814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0152s] [ 16%] 2024-08-08T21:00:14.9831413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 16%] 2024-08-08T21:00:14.9833033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 16%] 2024-08-08T21:00:14.9834634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0111s] [ 16%] 2024-08-08T21:00:14.9836219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0121s] [ 16%] 2024-08-08T21:00:14.9837825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 16%] 2024-08-08T21:00:14.9839422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 16%] 2024-08-08T21:00:14.9841089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0152s] [ 16%] 2024-08-08T21:00:14.9842756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0153s] [ 16%] 2024-08-08T21:00:14.9844363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 16%] 2024-08-08T21:00:14.9845960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0096s] [ 16%] 2024-08-08T21:00:14.9847597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0120s] [ 16%] 2024-08-08T21:00:14.9849237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0132s] [ 16%] 2024-08-08T21:00:14.9850832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 16%] 2024-08-08T21:00:14.9852467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 16%] 2024-08-08T21:00:14.9854061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0162s] [ 16%] 2024-08-08T21:00:14.9855683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0163s] [ 16%] 2024-08-08T21:00:14.9857270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0107s] [ 16%] 2024-08-08T21:00:14.9859254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0106s] [ 16%] 2024-08-08T21:00:14.9860871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0108s] [ 16%] 2024-08-08T21:00:14.9862476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0112s] [ 16%] 2024-08-08T21:00:14.9864041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 16%] 2024-08-08T21:00:14.9865742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 16%] 2024-08-08T21:00:14.9867404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0146s] [ 16%] 2024-08-08T21:00:14.9869016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0150s] [ 16%] 2024-08-08T21:00:14.9870668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 16%] 2024-08-08T21:00:14.9872331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 16%] 2024-08-08T21:00:14.9873925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 16%] 2024-08-08T21:00:14.9875509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 16%] 2024-08-08T21:00:14.9877106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 16%] 2024-08-08T21:00:14.9878703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 16%] 2024-08-08T21:00:14.9880301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 16%] 2024-08-08T21:00:14.9881903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 16%] 2024-08-08T21:00:14.9883521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 16%] 2024-08-08T21:00:14.9885127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 16%] 2024-08-08T21:00:14.9886720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 16%] 2024-08-08T21:00:14.9888355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 16%] 2024-08-08T21:00:14.9889983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 16%] 2024-08-08T21:00:14.9891581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 16%] 2024-08-08T21:00:14.9893158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 16%] 2024-08-08T21:00:14.9894824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 16%] 2024-08-08T21:00:14.9896464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 16%] 2024-08-08T21:00:14.9898077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 16%] 2024-08-08T21:00:14.9899656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 16%] 2024-08-08T21:00:14.9901267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 16%] 2024-08-08T21:00:14.9902840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 16%] 2024-08-08T21:00:14.9904438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 16%] 2024-08-08T21:00:14.9906024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 16%] 2024-08-08T21:00:14.9907626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 16%] 2024-08-08T21:00:14.9909225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 16%] 2024-08-08T21:00:14.9910821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 16%] 2024-08-08T21:00:14.9912457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 16%] 2024-08-08T21:00:14.9914097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 16%] 2024-08-08T21:00:14.9915671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 16%] 2024-08-08T21:00:14.9917295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 16%] 2024-08-08T21:00:14.9918950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 16%] 2024-08-08T21:00:14.9920530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 16%] 2024-08-08T21:00:14.9922112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 16%] 2024-08-08T21:00:14.9923700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 16%] 2024-08-08T21:00:14.9925277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 16%] 2024-08-08T21:00:14.9926868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 16%] 2024-08-08T21:00:14.9928428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 16%] 2024-08-08T21:00:14.9930026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 16%] 2024-08-08T21:00:14.9931603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 16%] 2024-08-08T21:00:14.9933197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 16%] 2024-08-08T21:00:14.9934811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 16%] 2024-08-08T21:00:14.9936469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 16%] 2024-08-08T21:00:14.9938025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 16%] 2024-08-08T21:00:14.9939614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 16%] 2024-08-08T21:00:14.9941223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 16%] 2024-08-08T21:00:14.9942914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 16%] 2024-08-08T21:00:14.9944503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 16%] 2024-08-08T21:00:14.9946083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 16%] 2024-08-08T21:00:14.9947678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 16%] 2024-08-08T21:00:14.9949258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 16%] 2024-08-08T21:00:14.9950837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 16%] 2024-08-08T21:00:14.9952460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 16%] 2024-08-08T21:00:14.9954045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 17%] 2024-08-08T21:00:14.9955639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 17%] 2024-08-08T21:00:14.9957205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:14.9959312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 17%] 2024-08-08T21:00:14.9960996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 17%] 2024-08-08T21:00:14.9962601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 17%] 2024-08-08T21:00:14.9964161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 17%] 2024-08-08T21:00:14.9965818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 17%] 2024-08-08T21:00:14.9967452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 17%] 2024-08-08T21:00:14.9969044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 17%] 2024-08-08T21:00:14.9970622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:14.9972225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 17%] 2024-08-08T21:00:14.9973787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:14.9975378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 17%] 2024-08-08T21:00:14.9976961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 17%] 2024-08-08T21:00:14.9978556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 17%] 2024-08-08T21:00:14.9980145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 17%] 2024-08-08T21:00:14.9981775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 17%] 2024-08-08T21:00:14.9983425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 17%] 2024-08-08T21:00:14.9985033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 17%] 2024-08-08T21:00:14.9986616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:14.9988257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 17%] 2024-08-08T21:00:14.9989899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 17%] 2024-08-08T21:00:14.9991480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 17%] 2024-08-08T21:00:14.9993066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 17%] 2024-08-08T21:00:14.9994656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 17%] 2024-08-08T21:00:14.9996236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 17%] 2024-08-08T21:00:14.9997837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 17%] 2024-08-08T21:00:14.9999422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 17%] 2024-08-08T21:00:15.0001045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 17%] 2024-08-08T21:00:15.0002615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 17%] 2024-08-08T21:00:15.0004217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 17%] 2024-08-08T21:00:15.0005833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 17%] 2024-08-08T21:00:15.0007482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 17%] 2024-08-08T21:00:15.0009057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 17%] 2024-08-08T21:00:15.0010662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 17%] 2024-08-08T21:00:15.0012296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:15.0013955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 17%] 2024-08-08T21:00:15.0015524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 17%] 2024-08-08T21:00:15.0017109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 17%] 2024-08-08T21:00:15.0018696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 17%] 2024-08-08T21:00:15.0020268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 17%] 2024-08-08T21:00:15.0021854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 17%] 2024-08-08T21:00:15.0023441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 17%] 2024-08-08T21:00:15.0025039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:15.0026615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 17%] 2024-08-08T21:00:15.0028240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:15.0029869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 17%] 2024-08-08T21:00:15.0031461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 17%] 2024-08-08T21:00:15.0033041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 17%] 2024-08-08T21:00:15.0034673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 17%] 2024-08-08T21:00:15.0036334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 17%] 2024-08-08T21:00:15.0037919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:15.0039535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:15.0041120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0067s] [ 17%] 2024-08-08T21:00:15.0042726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 17%] 2024-08-08T21:00:15.0044296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 17%] 2024-08-08T21:00:15.0045896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 17%] 2024-08-08T21:00:15.0047486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 17%] 2024-08-08T21:00:15.0049103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 17%] 2024-08-08T21:00:15.0050677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 17%] 2024-08-08T21:00:15.0052367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:15.0054020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 17%] 2024-08-08T21:00:15.0055599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 17%] 2024-08-08T21:00:15.0057185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 17%] 2024-08-08T21:00:15.0059241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 17%] 2024-08-08T21:00:15.0060945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0082s] [ 17%] 2024-08-08T21:00:15.0062536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 17%] 2024-08-08T21:00:15.0064172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:15.0065752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 17%] 2024-08-08T21:00:15.0067338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 17%] 2024-08-08T21:00:15.0068919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 17%] 2024-08-08T21:00:15.0070505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 17%] 2024-08-08T21:00:15.0072097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 17%] 2024-08-08T21:00:15.0073666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 17%] 2024-08-08T21:00:15.0075340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 17%] 2024-08-08T21:00:15.0076980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 17%] 2024-08-08T21:00:15.0078582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 18%] 2024-08-08T21:00:15.0080147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 18%] 2024-08-08T21:00:15.0081794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 18%] 2024-08-08T21:00:15.0083423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 18%] 2024-08-08T21:00:15.0085016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 18%] 2024-08-08T21:00:15.0086588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 18%] 2024-08-08T21:00:15.0088172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 18%] 2024-08-08T21:00:15.0089763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 18%] 2024-08-08T21:00:15.0091338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 18%] 2024-08-08T21:00:15.0092924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 18%] 2024-08-08T21:00:15.0094514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 18%] 2024-08-08T21:00:15.0096088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0097658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0099296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 18%] 2024-08-08T21:00:15.0100930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 18%] 2024-08-08T21:00:15.0102508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 18%] 2024-08-08T21:00:15.0104090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 18%] 2024-08-08T21:00:15.0105706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0107347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 18%] 2024-08-08T21:00:15.0108905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0110504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0112083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 18%] 2024-08-08T21:00:15.0113688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 18%] 2024-08-08T21:00:15.0115243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 18%] 2024-08-08T21:00:15.0116844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 18%] 2024-08-08T21:00:15.0118417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0120018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0121632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0123273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0124981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 18%] 2024-08-08T21:00:15.0126439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 18%] 2024-08-08T21:00:15.0128066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 18%] 2024-08-08T21:00:15.0129704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 18%] 2024-08-08T21:00:15.0131289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0132869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0134464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0136050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 18%] 2024-08-08T21:00:15.0137642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 18%] 2024-08-08T21:00:15.0139223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 18%] 2024-08-08T21:00:15.0140800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 18%] 2024-08-08T21:00:15.0142397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 18%] 2024-08-08T21:00:15.0143973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0145623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0147249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 18%] 2024-08-08T21:00:15.0148842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0150413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 18%] 2024-08-08T21:00:15.0152104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 18%] 2024-08-08T21:00:15.0153718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 18%] 2024-08-08T21:00:15.0155312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 18%] 2024-08-08T21:00:15.0156882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0158847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0160485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0162057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0163677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 18%] 2024-08-08T21:00:15.0165239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 18%] 2024-08-08T21:00:15.0166818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 18%] 2024-08-08T21:00:15.0168390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 18%] 2024-08-08T21:00:15.0170050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0171700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 18%] 2024-08-08T21:00:15.0173285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0174859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 18%] 2024-08-08T21:00:15.0176485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 18%] 2024-08-08T21:00:15.0178138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 18%] 2024-08-08T21:00:15.0179704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 18%] 2024-08-08T21:00:15.0181303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 18%] 2024-08-08T21:00:15.0182886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0084s] [ 18%] 2024-08-08T21:00:15.0184490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 18%] 2024-08-08T21:00:15.0186055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 18%] 2024-08-08T21:00:15.0187684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 18%] 2024-08-08T21:00:15.0189245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 18%] 2024-08-08T21:00:15.0190855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 18%] 2024-08-08T21:00:15.0192493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 18%] 2024-08-08T21:00:15.0194155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 18%] 2024-08-08T21:00:15.0195743Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0091s] [ 18%] 2024-08-08T21:00:15.0197329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0093s] [ 18%] 2024-08-08T21:00:15.0198961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 18%] 2024-08-08T21:00:15.0200595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 18%] 2024-08-08T21:00:15.0202188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0101s] [ 18%] 2024-08-08T21:00:15.0203780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 19%] 2024-08-08T21:00:15.0205389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0206993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0208585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0107s] [ 19%] 2024-08-08T21:00:15.0210175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0110s] [ 19%] 2024-08-08T21:00:15.0211751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0213347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 19%] 2024-08-08T21:00:15.0214918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0117s] [ 19%] 2024-08-08T21:00:15.0216580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0118s] [ 19%] 2024-08-08T21:00:15.0218226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 19%] 2024-08-08T21:00:15.0219833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 19%] 2024-08-08T21:00:15.0221400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 19%] 2024-08-08T21:00:15.0223041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0087s] [ 19%] 2024-08-08T21:00:15.0224652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0226241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0227817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0090s] [ 19%] 2024-08-08T21:00:15.0229414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0090s] [ 19%] 2024-08-08T21:00:15.0231005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0232585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0234172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 19%] 2024-08-08T21:00:15.0235763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 19%] 2024-08-08T21:00:15.0237339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 19%] 2024-08-08T21:00:15.0238958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0240597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0242177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0243761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0245384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 19%] 2024-08-08T21:00:15.0246999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 19%] 2024-08-08T21:00:15.0248588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0250145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0251781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 19%] 2024-08-08T21:00:15.0253355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 19%] 2024-08-08T21:00:15.0254955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0256525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0258124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0260054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0261660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0263305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0264955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0266539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0268127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0269777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 19%] 2024-08-08T21:00:15.0271418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0272998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0274582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0276182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 19%] 2024-08-08T21:00:15.0277749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 19%] 2024-08-08T21:00:15.0279332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 19%] 2024-08-08T21:00:15.0280921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 19%] 2024-08-08T21:00:15.0282493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0284108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0285694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0287345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0288918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0290508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0292123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0293793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0295359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0296949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0298515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0300103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 19%] 2024-08-08T21:00:15.0301689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 19%] 2024-08-08T21:00:15.0303265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0304857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0306445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 19%] 2024-08-08T21:00:15.0308027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 19%] 2024-08-08T21:00:15.0309653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0311304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 19%] 2024-08-08T21:00:15.0312863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 19%] 2024-08-08T21:00:15.0314457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0316136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0317761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 19%] 2024-08-08T21:00:15.0319358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 19%] 2024-08-08T21:00:15.0320927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 19%] 2024-08-08T21:00:15.0322531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 19%] 2024-08-08T21:00:15.0324094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 19%] 2024-08-08T21:00:15.0325686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 19%] 2024-08-08T21:00:15.0327256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 19%] 2024-08-08T21:00:15.0328847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 20%] 2024-08-08T21:00:15.0330412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 20%] 2024-08-08T21:00:15.0332008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 20%] 2024-08-08T21:00:15.0333619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 20%] 2024-08-08T21:00:15.0335250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 20%] 2024-08-08T21:00:15.0336860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0076s] [ 20%] 2024-08-08T21:00:15.0338508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 20%] 2024-08-08T21:00:15.0340169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 20%] 2024-08-08T21:00:15.0341769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 20%] 2024-08-08T21:00:15.0343385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0102s] [ 20%] 2024-08-08T21:00:15.0345004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 20%] 2024-08-08T21:00:15.0346627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0090s] [ 20%] 2024-08-08T21:00:15.0348238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 20%] 2024-08-08T21:00:15.0349847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 20%] 2024-08-08T21:00:15.0351459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 20%] 2024-08-08T21:00:15.0353101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0079s] [ 20%] 2024-08-08T21:00:15.0354720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 20%] 2024-08-08T21:00:15.0356359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0105s] [ 20%] 2024-08-08T21:00:15.0358032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 20%] 2024-08-08T21:00:15.0359993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0095s] [ 20%] 2024-08-08T21:00:15.0361642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0096s] [ 20%] 2024-08-08T21:00:15.0363322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0087s] [ 20%] 2024-08-08T21:00:15.0365012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 20%] 2024-08-08T21:00:15.0366599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0092s] [ 20%] 2024-08-08T21:00:15.0368240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 20%] 2024-08-08T21:00:15.0369819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0113s] [ 20%] 2024-08-08T21:00:15.0371440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0114s] [ 20%] 2024-08-08T21:00:15.0373030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0117s] [ 20%] 2024-08-08T21:00:15.0374641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0116s] [ 20%] 2024-08-08T21:00:15.0376245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 20%] 2024-08-08T21:00:15.0377835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 20%] 2024-08-08T21:00:15.0379503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 20%] 2024-08-08T21:00:15.0381180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 20%] 2024-08-08T21:00:15.0382789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0100s] [ 20%] 2024-08-08T21:00:15.0384383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 20%] 2024-08-08T21:00:15.0386046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 20%] 2024-08-08T21:00:15.0387696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 20%] 2024-08-08T21:00:15.0389305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0118s] [ 20%] 2024-08-08T21:00:15.0390901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0124s] [ 20%] 2024-08-08T21:00:15.0392489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0109s] [ 20%] 2024-08-08T21:00:15.0394116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0111s] [ 20%] 2024-08-08T21:00:15.0395703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0168s] [ 20%] 2024-08-08T21:00:15.0397326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0171s] [ 20%] 2024-08-08T21:00:15.0398925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0137s] [ 20%] 2024-08-08T21:00:15.0400545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0137s] [ 20%] 2024-08-08T21:00:15.0402136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0122s] [ 20%] 2024-08-08T21:00:15.0403794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0131s] [ 20%] 2024-08-08T21:00:15.0405435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0121s] [ 20%] 2024-08-08T21:00:15.0407054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0121s] [ 20%] 2024-08-08T21:00:15.0408644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0172s] [ 20%] 2024-08-08T21:00:15.0410319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0174s] [ 20%] 2024-08-08T21:00:15.0411953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0148s] [ 20%] 2024-08-08T21:00:15.0413555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0149s] [ 20%] 2024-08-08T21:00:15.0415155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0140s] [ 20%] 2024-08-08T21:00:15.0416760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0149s] [ 20%] 2024-08-08T21:00:15.0418371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0143s] [ 20%] 2024-08-08T21:00:15.0419964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0142s] [ 20%] 2024-08-08T21:00:15.0421575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0188s] [ 20%] 2024-08-08T21:00:15.0423184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0191s] [ 20%] 2024-08-08T21:00:15.0424792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0192s] [ 20%] 2024-08-08T21:00:15.0426445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0190s] [ 20%] 2024-08-08T21:00:15.0428080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0113s] [ 20%] 2024-08-08T21:00:15.0429683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0118s] [ 20%] 2024-08-08T21:00:15.0431267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0105s] [ 20%] 2024-08-08T21:00:15.0432919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0105s] [ 20%] 2024-08-08T21:00:15.0434561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0164s] [ 20%] 2024-08-08T21:00:15.0436175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0167s] [ 20%] 2024-08-08T21:00:15.0437766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0133s] [ 20%] 2024-08-08T21:00:15.0439388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 20%] 2024-08-08T21:00:15.0440974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 20%] 2024-08-08T21:00:15.0442580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 20%] 2024-08-08T21:00:15.0444167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 20%] 2024-08-08T21:00:15.0445770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 20%] 2024-08-08T21:00:15.0447370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 20%] 2024-08-08T21:00:15.0448968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 20%] 2024-08-08T21:00:15.0450620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 20%] 2024-08-08T21:00:15.0452315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 20%] 2024-08-08T21:00:15.0453916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 20%] 2024-08-08T21:00:15.0455541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 21%] 2024-08-08T21:00:15.0457164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 21%] 2024-08-08T21:00:15.0459207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 21%] 2024-08-08T21:00:15.0460852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 21%] 2024-08-08T21:00:15.0462458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 21%] 2024-08-08T21:00:15.0464073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 21%] 2024-08-08T21:00:15.0465674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 21%] 2024-08-08T21:00:15.0467252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 21%] 2024-08-08T21:00:15.0468872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 21%] 2024-08-08T21:00:15.0470465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 21%] 2024-08-08T21:00:15.0472078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 21%] 2024-08-08T21:00:15.0473749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0079s] [ 21%] 2024-08-08T21:00:15.0475470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 21%] 2024-08-08T21:00:15.0477059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 21%] 2024-08-08T21:00:15.0478672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 21%] 2024-08-08T21:00:15.0480332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 21%] 2024-08-08T21:00:15.0482004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 21%] 2024-08-08T21:00:15.0483646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 21%] 2024-08-08T21:00:15.0485185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 21%] 2024-08-08T21:00:15.0486776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 21%] 2024-08-08T21:00:15.0488468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 21%] 2024-08-08T21:00:15.0489971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 21%] 2024-08-08T21:00:15.0491571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 21%] 2024-08-08T21:00:15.0493175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 21%] 2024-08-08T21:00:15.0494765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 21%] 2024-08-08T21:00:15.0496355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 21%] 2024-08-08T21:00:15.0497983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 21%] 2024-08-08T21:00:15.0499643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 21%] 2024-08-08T21:00:15.0501232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 21%] 2024-08-08T21:00:15.0502867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 21%] 2024-08-08T21:00:15.0504536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 21%] 2024-08-08T21:00:15.0506111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0070s] [ 21%] 2024-08-08T21:00:15.0507718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 21%] 2024-08-08T21:00:15.0509292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 21%] 2024-08-08T21:00:15.0510898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 21%] 2024-08-08T21:00:15.0512477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 21%] 2024-08-08T21:00:15.0514088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 21%] 2024-08-08T21:00:15.0515679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 21%] 2024-08-08T21:00:15.0517303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 21%] 2024-08-08T21:00:15.0518884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 21%] 2024-08-08T21:00:15.0520519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 21%] 2024-08-08T21:00:15.0522164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 21%] 2024-08-08T21:00:15.0523749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 21%] 2024-08-08T21:00:15.0525340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 21%] 2024-08-08T21:00:15.0526979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 21%] 2024-08-08T21:00:15.0528640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 21%] 2024-08-08T21:00:15.0530236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 21%] 2024-08-08T21:00:15.0531832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 21%] 2024-08-08T21:00:15.0533420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 21%] 2024-08-08T21:00:15.0535007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 21%] 2024-08-08T21:00:15.0536583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 21%] 2024-08-08T21:00:15.0538165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 21%] 2024-08-08T21:00:15.0539777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 21%] 2024-08-08T21:00:15.0541347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 21%] 2024-08-08T21:00:15.0542943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 21%] 2024-08-08T21:00:15.0544585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0209s] [ 21%] 2024-08-08T21:00:15.0546259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0232s] [ 21%] 2024-08-08T21:00:15.0547876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0189s] [ 21%] 2024-08-08T21:00:15.0549523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0199s] [ 21%] 2024-08-08T21:00:15.0551169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0308s] [ 21%] 2024-08-08T21:00:15.0552843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0316s] [ 21%] 2024-08-08T21:00:15.0554438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0243s] [ 21%] 2024-08-08T21:00:15.0556058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0242s] [ 21%] 2024-08-08T21:00:15.0557671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0221s] [ 21%] 2024-08-08T21:00:15.0559611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.2000s] [ 21%] 2024-08-08T21:00:15.0561250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0221s] [ 21%] 2024-08-08T21:00:15.0562863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0223s] [ 21%] 2024-08-08T21:00:15.0564474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0320s] [ 21%] 2024-08-08T21:00:15.0566077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0331s] [ 21%] 2024-08-08T21:00:15.0567788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0276s] [ 21%] 2024-08-08T21:00:15.0569482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0273s] [ 21%] 2024-08-08T21:00:15.0571082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0253s] [ 21%] 2024-08-08T21:00:15.0572676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0282s] [ 21%] 2024-08-08T21:00:15.0574324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0262s] [ 21%] 2024-08-08T21:00:15.0576010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0263s] [ 21%] 2024-08-08T21:00:15.0577598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0353s] [ 21%] 2024-08-08T21:00:15.0579224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0353s] [ 21%] 2024-08-08T21:00:15.0580829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0348s] [ 21%] 2024-08-08T21:00:15.0582450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0343s] [ 22%] 2024-08-08T21:00:15.0584032Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0200s] [ 22%] 2024-08-08T21:00:15.0585646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0213s] [ 22%] 2024-08-08T21:00:15.0587241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0180s] [ 22%] 2024-08-08T21:00:15.0588856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0180s] [ 22%] 2024-08-08T21:00:15.0590485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0298s] [ 22%] 2024-08-08T21:00:15.0592153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0306s] [ 22%] 2024-08-08T21:00:15.0593752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.1903s] [ 22%] 2024-08-08T21:00:15.0595351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0235s] [ 22%] 2024-08-08T21:00:15.0596999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 22%] 2024-08-08T21:00:15.0598647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 22%] 2024-08-08T21:00:15.0600244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 22%] 2024-08-08T21:00:15.0601837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 22%] 2024-08-08T21:00:15.0603448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0081s] [ 22%] 2024-08-08T21:00:15.0605056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 22%] 2024-08-08T21:00:15.0606671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0079s] [ 22%] 2024-08-08T21:00:15.0608265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 22%] 2024-08-08T21:00:15.0609871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 22%] 2024-08-08T21:00:15.0611474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 22%] 2024-08-08T21:00:15.0613056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 22%] 2024-08-08T21:00:15.0614714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 22%] 2024-08-08T21:00:15.0616358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0085s] [ 22%] 2024-08-08T21:00:15.0617978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 22%] 2024-08-08T21:00:15.0619565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 22%] 2024-08-08T21:00:15.0621227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 22%] 2024-08-08T21:00:15.0622859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0073s] [ 22%] 2024-08-08T21:00:15.0624476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 22%] 2024-08-08T21:00:15.0626066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0083s] [ 22%] 2024-08-08T21:00:15.0627685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 22%] 2024-08-08T21:00:15.0629354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 22%] 2024-08-08T21:00:15.0630872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 22%] 2024-08-08T21:00:15.0632482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0095s] [ 22%] 2024-08-08T21:00:15.0634090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 22%] 2024-08-08T21:00:15.0635678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 22%] 2024-08-08T21:00:15.0637306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 22%] 2024-08-08T21:00:15.0638951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 22%] 2024-08-08T21:00:15.0640533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 22%] 2024-08-08T21:00:15.0642130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0080s] [ 22%] 2024-08-08T21:00:15.0643764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 22%] 2024-08-08T21:00:15.0645435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 22%] 2024-08-08T21:00:15.0647028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 22%] 2024-08-08T21:00:15.0648601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 22%] 2024-08-08T21:00:15.0650208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 22%] 2024-08-08T21:00:15.0651828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 22%] 2024-08-08T21:00:15.0653426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 22%] 2024-08-08T21:00:15.0655016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 22%] 2024-08-08T21:00:15.0656634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 22%] 2024-08-08T21:00:15.0658218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 22%] 2024-08-08T21:00:15.0660223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 22%] 2024-08-08T21:00:15.0661893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 22%] 2024-08-08T21:00:15.0663585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 22%] 2024-08-08T21:00:15.0665152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 22%] 2024-08-08T21:00:15.0666741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 22%] 2024-08-08T21:00:15.0668415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 22%] 2024-08-08T21:00:15.0670072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 22%] 2024-08-08T21:00:15.0671675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 22%] 2024-08-08T21:00:15.0673280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 22%] 2024-08-08T21:00:15.0674883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 22%] 2024-08-08T21:00:15.0676470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 22%] 2024-08-08T21:00:15.0678052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 22%] 2024-08-08T21:00:15.0679636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 22%] 2024-08-08T21:00:15.0681250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 22%] 2024-08-08T21:00:15.0682849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 22%] 2024-08-08T21:00:15.0684482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 22%] 2024-08-08T21:00:15.0686205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 22%] 2024-08-08T21:00:15.0687783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 22%] 2024-08-08T21:00:15.0689380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 22%] 2024-08-08T21:00:15.0690988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 22%] 2024-08-08T21:00:15.0692634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 22%] 2024-08-08T21:00:15.0694210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 22%] 2024-08-08T21:00:15.0695817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 22%] 2024-08-08T21:00:15.0697392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 22%] 2024-08-08T21:00:15.0699003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 22%] 2024-08-08T21:00:15.0700594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0089s] [ 22%] 2024-08-08T21:00:15.0702201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0093s] [ 22%] 2024-08-08T21:00:15.0703821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0095s] [ 22%] 2024-08-08T21:00:15.0705427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 22%] 2024-08-08T21:00:15.0707038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0122s] [ 22%] 2024-08-08T21:00:15.0708693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0125s] [ 23%] 2024-08-08T21:00:15.0710354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0113s] [ 23%] 2024-08-08T21:00:15.0711958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0113s] [ 23%] 2024-08-08T21:00:15.0713569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0097s] [ 23%] 2024-08-08T21:00:15.0715225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 23%] 2024-08-08T21:00:15.0716883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0103s] [ 23%] 2024-08-08T21:00:15.0718487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0105s] [ 23%] 2024-08-08T21:00:15.0720105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0126s] [ 23%] 2024-08-08T21:00:15.0721724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0131s] [ 23%] 2024-08-08T21:00:15.0723323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0118s] [ 23%] 2024-08-08T21:00:15.0724951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0118s] [ 23%] 2024-08-08T21:00:15.0726542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0110s] [ 23%] 2024-08-08T21:00:15.0728166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0114s] [ 23%] 2024-08-08T21:00:15.0729758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0121s] [ 23%] 2024-08-08T21:00:15.0731432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0120s] [ 23%] 2024-08-08T21:00:15.0733078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0141s] [ 23%] 2024-08-08T21:00:15.0734702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0140s] [ 23%] 2024-08-08T21:00:15.0736297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0148s] [ 23%] 2024-08-08T21:00:15.0737982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0148s] [ 23%] 2024-08-08T21:00:15.0739619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 23%] 2024-08-08T21:00:15.0741211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 23%] 2024-08-08T21:00:15.0742820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0093s] [ 23%] 2024-08-08T21:00:15.0744431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0091s] [ 23%] 2024-08-08T21:00:15.0746042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0120s] [ 23%] 2024-08-08T21:00:15.0747644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0122s] [ 23%] 2024-08-08T21:00:15.0749263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0111s] [ 23%] 2024-08-08T21:00:15.0750872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 23%] 2024-08-08T21:00:15.0752524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0143s] [ 23%] 2024-08-08T21:00:15.0754123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0153s] [ 23%] 2024-08-08T21:00:15.0755789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0151s] [ 23%] 2024-08-08T21:00:15.0757448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0146s] [ 23%] 2024-08-08T21:00:15.0759398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0203s] [ 23%] 2024-08-08T21:00:15.0761053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0209s] [ 23%] 2024-08-08T21:00:15.0762738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0181s] [ 23%] 2024-08-08T21:00:15.0764432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0177s] [ 23%] 2024-08-08T21:00:15.0766017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0152s] [ 23%] 2024-08-08T21:00:15.0767638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0165s] [ 23%] 2024-08-08T21:00:15.0769238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0163s] [ 23%] 2024-08-08T21:00:15.0770861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0162s] [ 23%] 2024-08-08T21:00:15.0772456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0212s] [ 23%] 2024-08-08T21:00:15.0774082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0214s] [ 23%] 2024-08-08T21:00:15.0775680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0194s] [ 23%] 2024-08-08T21:00:15.0777287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0203s] [ 23%] 2024-08-08T21:00:15.0778961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0178s] [ 23%] 2024-08-08T21:00:15.0780635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0189s] [ 23%] 2024-08-08T21:00:15.0782237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0195s] [ 23%] 2024-08-08T21:00:15.0783830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0194s] [ 23%] 2024-08-08T21:00:15.0785489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0238s] [ 23%] 2024-08-08T21:00:15.0787141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0239s] [ 23%] 2024-08-08T21:00:15.0788757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0247s] [ 23%] 2024-08-08T21:00:15.0790369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0250s] [ 23%] 2024-08-08T21:00:15.0791979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0140s] [ 23%] 2024-08-08T21:00:15.0793569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0144s] [ 23%] 2024-08-08T21:00:15.0795185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0143s] [ 23%] 2024-08-08T21:00:15.0796771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0144s] [ 23%] 2024-08-08T21:00:15.0798366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0200s] [ 23%] 2024-08-08T21:00:15.0799983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0207s] [ 23%] 2024-08-08T21:00:15.0801620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0174s] [ 23%] 2024-08-08T21:00:15.0803294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0176s] [ 23%] 2024-08-08T21:00:15.0804881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 23%] 2024-08-08T21:00:15.0806496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 23%] 2024-08-08T21:00:15.0808123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 23%] 2024-08-08T21:00:15.0809783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 23%] 2024-08-08T21:00:15.0811370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0084s] [ 23%] 2024-08-08T21:00:15.0812987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0085s] [ 23%] 2024-08-08T21:00:15.0814583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0087s] [ 23%] 2024-08-08T21:00:15.0816188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 23%] 2024-08-08T21:00:15.0817790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0070s] [ 23%] 2024-08-08T21:00:15.0819389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 23%] 2024-08-08T21:00:15.0821001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 23%] 2024-08-08T21:00:15.0822598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 23%] 2024-08-08T21:00:15.0824196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0086s] [ 23%] 2024-08-08T21:00:15.0825838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 23%] 2024-08-08T21:00:15.0827515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0077s] [ 23%] 2024-08-08T21:00:15.0829110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 23%] 2024-08-08T21:00:15.0830712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0076s] [ 23%] 2024-08-08T21:00:15.0832347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 23%] 2024-08-08T21:00:15.0833988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0078s] [ 24%] 2024-08-08T21:00:15.0835598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 24%] 2024-08-08T21:00:15.0837186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0092s] [ 24%] 2024-08-08T21:00:15.0838804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0093s] [ 24%] 2024-08-08T21:00:15.0840386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0097s] [ 24%] 2024-08-08T21:00:15.0842024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0094s] [ 24%] 2024-08-08T21:00:15.0843580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 24%] 2024-08-08T21:00:15.0845192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 24%] 2024-08-08T21:00:15.0846757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 24%] 2024-08-08T21:00:15.0848397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 24%] 2024-08-08T21:00:15.0850027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0083s] [ 24%] 2024-08-08T21:00:15.0851623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 24%] 2024-08-08T21:00:15.0853266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 24%] 2024-08-08T21:00:15.0854905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 24%] 2024-08-08T21:00:15.0856567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 24%] 2024-08-08T21:00:15.0858143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 24%] 2024-08-08T21:00:15.0860081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 24%] 2024-08-08T21:00:15.0861689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 24%] 2024-08-08T21:00:15.0863296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 24%] 2024-08-08T21:00:15.0864882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 24%] 2024-08-08T21:00:15.0866491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 24%] 2024-08-08T21:00:15.0868097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 24%] 2024-08-08T21:00:15.0869678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 24%] 2024-08-08T21:00:15.0871281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 24%] 2024-08-08T21:00:15.0872932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0068s] [ 24%] 2024-08-08T21:00:15.0874607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 24%] 2024-08-08T21:00:15.0876188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0083s] [ 24%] 2024-08-08T21:00:15.0877797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 24%] 2024-08-08T21:00:15.0879445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 24%] 2024-08-08T21:00:15.0881133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 24%] 2024-08-08T21:00:15.0882710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0071s] [ 24%] 2024-08-08T21:00:15.0884320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 24%] 2024-08-08T21:00:15.0885902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 24%] 2024-08-08T21:00:15.0887480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 24%] 2024-08-08T21:00:15.0889073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0090s] [ 24%] 2024-08-08T21:00:15.0890667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0090s] [ 24%] 2024-08-08T21:00:15.0892279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0080s] [ 24%] 2024-08-08T21:00:15.0893883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 24%] 2024-08-08T21:00:15.0895523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 24%] 2024-08-08T21:00:15.0897154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 24%] 2024-08-08T21:00:15.0898742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 24%] 2024-08-08T21:00:15.0900319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 24%] 2024-08-08T21:00:15.0901958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 24%] 2024-08-08T21:00:15.0903595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 24%] 2024-08-08T21:00:15.0905164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 24%] 2024-08-08T21:00:15.0906761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 24%] 2024-08-08T21:00:15.0908354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0258s] [ 24%] 2024-08-08T21:00:15.0910002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1991s] [ 24%] 2024-08-08T21:00:15.0911579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0264s] [ 24%] 2024-08-08T21:00:15.0913205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0245s] [ 24%] 2024-08-08T21:00:15.0914810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0375s] [ 24%] 2024-08-08T21:00:15.0916441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0385s] [ 24%] 2024-08-08T21:00:15.0918040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0306s] [ 24%] 2024-08-08T21:00:15.0919735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0315s] [ 24%] 2024-08-08T21:00:15.0921386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0288s] [ 24%] 2024-08-08T21:00:15.0923000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0300s] [ 24%] 2024-08-08T21:00:15.0924599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0300s] [ 24%] 2024-08-08T21:00:15.0926254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0292s] [ 24%] 2024-08-08T21:00:15.0927913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0384s] [ 24%] 2024-08-08T21:00:15.0929521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0400s] [ 24%] 2024-08-08T21:00:15.0931147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0354s] [ 24%] 2024-08-08T21:00:15.0932767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0345s] [ 24%] 2024-08-08T21:00:15.0934372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0341s] [ 24%] 2024-08-08T21:00:15.0935970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0346s] [ 24%] 2024-08-08T21:00:15.0937583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0358s] [ 24%] 2024-08-08T21:00:15.0939194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0356s] [ 24%] 2024-08-08T21:00:15.0940802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.2085s] [ 24%] 2024-08-08T21:00:15.0942450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0442s] [ 24%] 2024-08-08T21:00:15.0944119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0450s] [ 24%] 2024-08-08T21:00:15.0945731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0440s] [ 24%] 2024-08-08T21:00:15.0947316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0246s] [ 24%] 2024-08-08T21:00:15.0948973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0263s] [ 24%] 2024-08-08T21:00:15.0950606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0232s] [ 24%] 2024-08-08T21:00:15.0952266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0250s] [ 24%] 2024-08-08T21:00:15.0953861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0366s] [ 24%] 2024-08-08T21:00:15.0955495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0376s] [ 24%] 2024-08-08T21:00:15.0957091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0302s] [ 24%] 2024-08-08T21:00:15.0959121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0301s] [ 24%] 2024-08-08T21:00:15.0960752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0071s] [ 25%] 2024-08-08T21:00:15.0962359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 25%] 2024-08-08T21:00:15.0963963Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 25%] 2024-08-08T21:00:15.0965557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 25%] 2024-08-08T21:00:15.0967246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0088s] [ 25%] 2024-08-08T21:00:15.0968932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 25%] 2024-08-08T21:00:15.0970534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0085s] [ 25%] 2024-08-08T21:00:15.0972128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 25%] 2024-08-08T21:00:15.0973788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0080s] [ 25%] 2024-08-08T21:00:15.0975445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 25%] 2024-08-08T21:00:15.0977044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 25%] 2024-08-08T21:00:15.0978641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 25%] 2024-08-08T21:00:15.0980239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 25%] 2024-08-08T21:00:15.0981859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 25%] 2024-08-08T21:00:15.0983445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0092s] [ 25%] 2024-08-08T21:00:15.0985065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 25%] 2024-08-08T21:00:15.0986655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0085s] [ 25%] 2024-08-08T21:00:15.0988261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 25%] 2024-08-08T21:00:15.0989890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0094s] [ 25%] 2024-08-08T21:00:15.0991558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 25%] 2024-08-08T21:00:15.0993147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0095s] [ 25%] 2024-08-08T21:00:15.0994763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 25%] 2024-08-08T21:00:15.0996393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0105s] [ 25%] 2024-08-08T21:00:15.0998056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0103s] [ 25%] 2024-08-08T21:00:15.0999649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0064s] [ 25%] 2024-08-08T21:00:15.1001236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 25%] 2024-08-08T21:00:15.1002833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 25%] 2024-08-08T21:00:15.1004410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 25%] 2024-08-08T21:00:15.1006010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0084s] [ 25%] 2024-08-08T21:00:15.1007601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 25%] 2024-08-08T21:00:15.1009229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0078s] [ 25%] 2024-08-08T21:00:15.1010801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 25%] 2024-08-08T21:00:15.1012402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 25%] 2024-08-08T21:00:15.1014019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 25%] 2024-08-08T21:00:15.1015644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 25%] 2024-08-08T21:00:15.1017240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 25%] 2024-08-08T21:00:15.1018822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0079s] [ 25%] 2024-08-08T21:00:15.1020475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 25%] 2024-08-08T21:00:15.1022108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 25%] 2024-08-08T21:00:15.1023717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 25%] 2024-08-08T21:00:15.1025294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 25%] 2024-08-08T21:00:15.1026905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 25%] 2024-08-08T21:00:15.1028472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 25%] 2024-08-08T21:00:15.1030071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 25%] 2024-08-08T21:00:15.1031664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0082s] [ 25%] 2024-08-08T21:00:15.1033279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 25%] 2024-08-08T21:00:15.1034866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 25%] 2024-08-08T21:00:15.1036509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 25%] 2024-08-08T21:00:15.1038148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 25%] 2024-08-08T21:00:15.1039736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 25%] 2024-08-08T21:00:15.1041320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0068s] [ 25%] 2024-08-08T21:00:15.1042946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 25%] 2024-08-08T21:00:15.1044597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0088s] [ 25%] 2024-08-08T21:00:15.1046183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 25%] 2024-08-08T21:00:15.1047785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0079s] [ 25%] 2024-08-08T21:00:15.1049385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 25%] 2024-08-08T21:00:15.1050975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 25%] 2024-08-08T21:00:15.1052592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 25%] 2024-08-08T21:00:15.1054163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 25%] 2024-08-08T21:00:15.1055788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 25%] 2024-08-08T21:00:15.1057340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 25%] 2024-08-08T21:00:15.1059291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 25%] 2024-08-08T21:00:15.1061029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 25%] 2024-08-08T21:00:15.1062715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_152_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 25%] 2024-08-08T21:00:15.1064305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0117s] [ 25%] 2024-08-08T21:00:15.1065929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0122s] [ 25%] 2024-08-08T21:00:15.1067589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0114s] [ 25%] 2024-08-08T21:00:15.1069281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0116s] [ 25%] 2024-08-08T21:00:15.1070874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0168s] [ 25%] 2024-08-08T21:00:15.1072489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0168s] [ 25%] 2024-08-08T21:00:15.1074117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0145s] [ 25%] 2024-08-08T21:00:15.1075725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0146s] [ 25%] 2024-08-08T21:00:15.1077330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0121s] [ 25%] 2024-08-08T21:00:15.1078941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0131s] [ 25%] 2024-08-08T21:00:15.1080554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0132s] [ 25%] 2024-08-08T21:00:15.1082152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0130s] [ 25%] 2024-08-08T21:00:15.1083813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1884s] [ 25%] 2024-08-08T21:00:15.1085482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0176s] [ 25%] 2024-08-08T21:00:15.1087089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0162s] [ 26%] 2024-08-08T21:00:15.1088694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0163s] [ 26%] 2024-08-08T21:00:15.1090349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0142s] [ 26%] 2024-08-08T21:00:15.1092024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0152s] [ 26%] 2024-08-08T21:00:15.1093614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0156s] [ 26%] 2024-08-08T21:00:15.1095237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0159s] [ 26%] 2024-08-08T21:00:15.1096840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0196s] [ 26%] 2024-08-08T21:00:15.1098473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0195s] [ 26%] 2024-08-08T21:00:15.1100065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0206s] [ 26%] 2024-08-08T21:00:15.1101695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0204s] [ 26%] 2024-08-08T21:00:15.1103285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0114s] [ 26%] 2024-08-08T21:00:15.1104891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0116s] [ 26%] 2024-08-08T21:00:15.1106531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0116s] [ 26%] 2024-08-08T21:00:15.1108177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0115s] [ 26%] 2024-08-08T21:00:15.1109788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0166s] [ 26%] 2024-08-08T21:00:15.1111390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0166s] [ 26%] 2024-08-08T21:00:15.1113099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0147s] [ 26%] 2024-08-08T21:00:15.1114701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0147s] [ 26%] 2024-08-08T21:00:15.1116307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0197s] [ 26%] 2024-08-08T21:00:15.1117907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0214s] [ 26%] 2024-08-08T21:00:15.1119523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0199s] [ 26%] 2024-08-08T21:00:15.1121138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0195s] [ 26%] 2024-08-08T21:00:15.1122753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0298s] [ 26%] 2024-08-08T21:00:15.1124360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0302s] [ 26%] 2024-08-08T21:00:15.1125964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0254s] [ 26%] 2024-08-08T21:00:15.1127591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0251s] [ 26%] 2024-08-08T21:00:15.1129221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0212s] [ 26%] 2024-08-08T21:00:15.1130842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0228s] [ 26%] 2024-08-08T21:00:15.1132489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0223s] [ 26%] 2024-08-08T21:00:15.1134104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0220s] [ 26%] 2024-08-08T21:00:15.1135702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1986s] [ 26%] 2024-08-08T21:00:15.1137370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0307s] [ 26%] 2024-08-08T21:00:15.1139008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0266s] [ 26%] 2024-08-08T21:00:15.1140637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0272s] [ 26%] 2024-08-08T21:00:15.1142232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0235s] [ 26%] 2024-08-08T21:00:15.1143862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0258s] [ 26%] 2024-08-08T21:00:15.1145450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0255s] [ 26%] 2024-08-08T21:00:15.1147042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0248s] [ 26%] 2024-08-08T21:00:15.1148655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0338s] [ 26%] 2024-08-08T21:00:15.1150273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0335s] [ 26%] 2024-08-08T21:00:15.1151928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0350s] [ 26%] 2024-08-08T21:00:15.1153586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0353s] [ 26%] 2024-08-08T21:00:15.1155246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0191s] [ 26%] 2024-08-08T21:00:15.1156849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0197s] [ 26%] 2024-08-08T21:00:15.1158790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0182s] [ 26%] 2024-08-08T21:00:15.1160487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0183s] [ 26%] 2024-08-08T21:00:15.1162170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0286s] [ 26%] 2024-08-08T21:00:15.1163773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0293s] [ 26%] 2024-08-08T21:00:15.1165365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0236s] [ 26%] 2024-08-08T21:00:15.1166986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0240s] [ 26%] 2024-08-08T21:00:15.1168579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 26%] 2024-08-08T21:00:15.1170187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 26%] 2024-08-08T21:00:15.1171773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0079s] [ 26%] 2024-08-08T21:00:15.1173390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 26%] 2024-08-08T21:00:15.1174978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0108s] [ 26%] 2024-08-08T21:00:15.1176602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 26%] 2024-08-08T21:00:15.1178252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 26%] 2024-08-08T21:00:15.1179962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0099s] [ 26%] 2024-08-08T21:00:15.1181545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0087s] [ 26%] 2024-08-08T21:00:15.1183140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1804s] [ 26%] 2024-08-08T21:00:15.1184790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0082s] [ 26%] 2024-08-08T21:00:15.1186431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 26%] 2024-08-08T21:00:15.1188033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0114s] [ 26%] 2024-08-08T21:00:15.1189636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0114s] [ 26%] 2024-08-08T21:00:15.1191244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 26%] 2024-08-08T21:00:15.1192840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0098s] [ 26%] 2024-08-08T21:00:15.1194443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0095s] [ 26%] 2024-08-08T21:00:15.1196043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 26%] 2024-08-08T21:00:15.1197653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0103s] [ 26%] 2024-08-08T21:00:15.1199243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0103s] [ 26%] 2024-08-08T21:00:15.1200871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0126s] [ 26%] 2024-08-08T21:00:15.1202546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0124s] [ 26%] 2024-08-08T21:00:15.1204132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0122s] [ 26%] 2024-08-08T21:00:15.1205742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0120s] [ 26%] 2024-08-08T21:00:15.1207363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0076s] [ 26%] 2024-08-08T21:00:15.1209026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 26%] 2024-08-08T21:00:15.1210591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0076s] [ 26%] 2024-08-08T21:00:15.1212190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 26%] 2024-08-08T21:00:15.1213776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0105s] [ 27%] 2024-08-08T21:00:15.1215383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 27%] 2024-08-08T21:00:15.1216966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 27%] 2024-08-08T21:00:15.1218573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 27%] 2024-08-08T21:00:15.1220176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 27%] 2024-08-08T21:00:15.1221764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 27%] 2024-08-08T21:00:15.1223349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0076s] [ 27%] 2024-08-08T21:00:15.1224979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 27%] 2024-08-08T21:00:15.1226635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0100s] [ 27%] 2024-08-08T21:00:15.1228228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 27%] 2024-08-08T21:00:15.1229834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0087s] [ 27%] 2024-08-08T21:00:15.1231479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 27%] 2024-08-08T21:00:15.1233116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0082s] [ 27%] 2024-08-08T21:00:15.1234696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 27%] 2024-08-08T21:00:15.1236266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 27%] 2024-08-08T21:00:15.1237868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 27%] 2024-08-08T21:00:15.1239446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0096s] [ 27%] 2024-08-08T21:00:15.1241052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 27%] 2024-08-08T21:00:15.1242638Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0076s] [ 27%] 2024-08-08T21:00:15.1244260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 27%] 2024-08-08T21:00:15.1245854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0085s] [ 27%] 2024-08-08T21:00:15.1247483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 27%] 2024-08-08T21:00:15.1249104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0085s] [ 27%] 2024-08-08T21:00:15.1250701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0083s] [ 27%] 2024-08-08T21:00:15.1252318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0104s] [ 27%] 2024-08-08T21:00:15.1253973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 27%] 2024-08-08T21:00:15.1255612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0089s] [ 27%] 2024-08-08T21:00:15.1257206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0094s] [ 27%] 2024-08-08T21:00:15.1259205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0068s] [ 27%] 2024-08-08T21:00:15.1260804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 27%] 2024-08-08T21:00:15.1262389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 27%] 2024-08-08T21:00:15.1263967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 27%] 2024-08-08T21:00:15.1265565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 27%] 2024-08-08T21:00:15.1267155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 27%] 2024-08-08T21:00:15.1268739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0079s] [ 27%] 2024-08-08T21:00:15.1270318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 27%] 2024-08-08T21:00:15.1272023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0356s] [ 27%] 2024-08-08T21:00:15.1273702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0389s] [ 27%] 2024-08-08T21:00:15.1275298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0342s] [ 27%] 2024-08-08T21:00:15.1276927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0340s] [ 27%] 2024-08-08T21:00:15.1278598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0552s] [ 27%] 2024-08-08T21:00:15.1280290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0571s] [ 27%] 2024-08-08T21:00:15.1281891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0450s] [ 27%] 2024-08-08T21:00:15.1283519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0450s] [ 27%] 2024-08-08T21:00:15.1285117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0380s] [ 27%] 2024-08-08T21:00:15.1286732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0414s] [ 27%] 2024-08-08T21:00:15.1288322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.2037s] [ 27%] 2024-08-08T21:00:15.1289948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0387s] [ 27%] 2024-08-08T21:00:15.1291546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0569s] [ 27%] 2024-08-08T21:00:15.1293159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0596s] [ 27%] 2024-08-08T21:00:15.1296417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0503s] [ 27%] 2024-08-08T21:00:15.1299618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0509s] [ 27%] 2024-08-08T21:00:15.1302789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0435s] [ 27%] 2024-08-08T21:00:15.1305942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0475s] [ 27%] 2024-08-08T21:00:15.1309065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0468s] [ 27%] 2024-08-08T21:00:15.1312259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0464s] [ 27%] 2024-08-08T21:00:15.1315440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0618s] [ 27%] 2024-08-08T21:00:15.1318534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.2307s] [ 27%] 2024-08-08T21:00:15.1321663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0650s] [ 27%] 2024-08-08T21:00:15.1324853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0660s] [ 27%] 2024-08-08T21:00:15.1327911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0356s] [ 27%] 2024-08-08T21:00:15.1331017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0374s] [ 27%] 2024-08-08T21:00:15.1334128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0335s] [ 27%] 2024-08-08T21:00:15.1337247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0328s] [ 27%] 2024-08-08T21:00:15.1340341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0550s] [ 27%] 2024-08-08T21:00:15.1343536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0564s] [ 27%] 2024-08-08T21:00:15.1347422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0442s] [ 27%] 2024-08-08T21:00:15.1350649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0437s] [ 27%] 2024-08-08T21:00:15.1353747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0086s] [ 27%] 2024-08-08T21:00:15.1356912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0087s] [ 27%] 2024-08-08T21:00:15.1360416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0091s] [ 27%] 2024-08-08T21:00:15.1363527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 27%] 2024-08-08T21:00:15.1366649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0112s] [ 27%] 2024-08-08T21:00:15.1369759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 27%] 2024-08-08T21:00:15.1372872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0105s] [ 27%] 2024-08-08T21:00:15.1375990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0105s] [ 27%] 2024-08-08T21:00:15.1379095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0092s] [ 27%] 2024-08-08T21:00:15.1382198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1782s] [ 27%] 2024-08-08T21:00:15.1385297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0100s] [ 28%] 2024-08-08T21:00:15.1388487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0098s] [ 28%] 2024-08-08T21:00:15.1391699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0120s] [ 28%] 2024-08-08T21:00:15.1394810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0120s] [ 28%] 2024-08-08T21:00:15.1397918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0114s] [ 28%] 2024-08-08T21:00:15.1401107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0117s] [ 28%] 2024-08-08T21:00:15.1404313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0104s] [ 28%] 2024-08-08T21:00:15.1407397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0108s] [ 28%] 2024-08-08T21:00:15.1410506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0112s] [ 28%] 2024-08-08T21:00:15.1413619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0116s] [ 28%] 2024-08-08T21:00:15.1416732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0134s] [ 28%] 2024-08-08T21:00:15.1419844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0128s] [ 28%] 2024-08-08T21:00:15.1422962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0138s] [ 28%] 2024-08-08T21:00:15.1426096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0137s] [ 28%] 2024-08-08T21:00:15.1429168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0081s] [ 28%] 2024-08-08T21:00:15.1432304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 28%] 2024-08-08T21:00:15.1435456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0084s] [ 28%] 2024-08-08T21:00:15.1438548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 28%] 2024-08-08T21:00:15.1441639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0107s] [ 28%] 2024-08-08T21:00:15.1444766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0109s] [ 28%] 2024-08-08T21:00:15.1447922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0100s] [ 28%] 2024-08-08T21:00:15.1451028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0096s] [ 28%] 2024-08-08T21:00:15.1454170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 28%] 2024-08-08T21:00:15.1457274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 28%] 2024-08-08T21:00:15.1460780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 28%] 2024-08-08T21:00:15.1463859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 28%] 2024-08-08T21:00:15.1466958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0103s] [ 28%] 2024-08-08T21:00:15.1470067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0100s] [ 28%] 2024-08-08T21:00:15.1473174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0087s] [ 28%] 2024-08-08T21:00:15.1476279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 28%] 2024-08-08T21:00:15.1479461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0084s] [ 28%] 2024-08-08T21:00:15.1482631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 28%] 2024-08-08T21:00:15.1485720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0071s] [ 28%] 2024-08-08T21:00:15.1488843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 28%] 2024-08-08T21:00:15.1491968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0100s] [ 28%] 2024-08-08T21:00:15.1495154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0104s] [ 28%] 2024-08-08T21:00:15.1498246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 28%] 2024-08-08T21:00:15.1501355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 28%] 2024-08-08T21:00:15.1504471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 28%] 2024-08-08T21:00:15.1507817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0085s] [ 28%] 2024-08-08T21:00:15.1510944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0082s] [ 28%] 2024-08-08T21:00:15.1514044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 28%] 2024-08-08T21:00:15.1517134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0109s] [ 28%] 2024-08-08T21:00:15.1520231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0107s] [ 28%] 2024-08-08T21:00:15.1523404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 28%] 2024-08-08T21:00:15.1526574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 28%] 2024-08-08T21:00:15.1529682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 28%] 2024-08-08T21:00:15.1532774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 28%] 2024-08-08T21:00:15.1535876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 28%] 2024-08-08T21:00:15.1539085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 28%] 2024-08-08T21:00:15.1542195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 28%] 2024-08-08T21:00:15.1545263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 28%] 2024-08-08T21:00:15.1548357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0076s] [ 28%] 2024-08-08T21:00:15.1551457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 28%] 2024-08-08T21:00:15.1554580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 28%] 2024-08-08T21:00:15.1557671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 28%] 2024-08-08T21:00:15.1562923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 28%] 2024-08-08T21:00:15.1566037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 28%] 2024-08-08T21:00:15.1569127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 28%] 2024-08-08T21:00:15.1572328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 28%] 2024-08-08T21:00:15.1575526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 28%] 2024-08-08T21:00:15.1578634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 28%] 2024-08-08T21:00:15.1581730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 28%] 2024-08-08T21:00:15.1584905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 28%] 2024-08-08T21:00:15.1588077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 28%] 2024-08-08T21:00:15.1591144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 28%] 2024-08-08T21:00:15.1594240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 28%] 2024-08-08T21:00:15.1597351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 28%] 2024-08-08T21:00:15.1600455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 28%] 2024-08-08T21:00:15.1603555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 28%] 2024-08-08T21:00:15.1606656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 28%] 2024-08-08T21:00:15.1609742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 28%] 2024-08-08T21:00:15.1612836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 28%] 2024-08-08T21:00:15.1615967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 28%] 2024-08-08T21:00:15.1619121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 28%] 2024-08-08T21:00:15.1622234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 28%] 2024-08-08T21:00:15.1625341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 28%] 2024-08-08T21:00:15.1628475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 28%] 2024-08-08T21:00:15.1631627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 29%] 2024-08-08T21:00:15.1634713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 29%] 2024-08-08T21:00:15.1637790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 29%] 2024-08-08T21:00:15.1640881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 29%] 2024-08-08T21:00:15.1643973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1647046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1650138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 29%] 2024-08-08T21:00:15.1653282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 29%] 2024-08-08T21:00:15.1656386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 29%] 2024-08-08T21:00:15.1660062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 29%] 2024-08-08T21:00:15.1663299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 29%] 2024-08-08T21:00:15.1666477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 29%] 2024-08-08T21:00:15.1669584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0082s] [ 29%] 2024-08-08T21:00:15.1672687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 29%] 2024-08-08T21:00:15.1675864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 29%] 2024-08-08T21:00:15.1679048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1682132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1685247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 29%] 2024-08-08T21:00:15.1688349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 29%] 2024-08-08T21:00:15.1691439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 29%] 2024-08-08T21:00:15.1694539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 29%] 2024-08-08T21:00:15.1697650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 29%] 2024-08-08T21:00:15.1700751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 29%] 2024-08-08T21:00:15.1703854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1707229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0080s] [ 29%] 2024-08-08T21:00:15.1710464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 29%] 2024-08-08T21:00:15.1713559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 29%] 2024-08-08T21:00:15.1716645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 29%] 2024-08-08T21:00:15.1719774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0090s] [ 29%] 2024-08-08T21:00:15.1722934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 29%] 2024-08-08T21:00:15.1726034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 29%] 2024-08-08T21:00:15.1729138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 29%] 2024-08-08T21:00:15.1732234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0064s] [ 29%] 2024-08-08T21:00:15.1735338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 29%] 2024-08-08T21:00:15.1738399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 29%] 2024-08-08T21:00:15.1741468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 29%] 2024-08-08T21:00:15.1744564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 29%] 2024-08-08T21:00:15.1747652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 29%] 2024-08-08T21:00:15.1750740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1753910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 29%] 2024-08-08T21:00:15.1757069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 29%] 2024-08-08T21:00:15.1760445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 29%] 2024-08-08T21:00:15.1763534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 29%] 2024-08-08T21:00:15.1766701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 29%] 2024-08-08T21:00:15.1769879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1772945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 29%] 2024-08-08T21:00:15.1776043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 29%] 2024-08-08T21:00:15.1779144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 29%] 2024-08-08T21:00:15.1782230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 29%] 2024-08-08T21:00:15.1785307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 29%] 2024-08-08T21:00:15.1788386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 29%] 2024-08-08T21:00:15.1791456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 29%] 2024-08-08T21:00:15.1794532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 29%] 2024-08-08T21:00:15.1797611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 29%] 2024-08-08T21:00:15.1800766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 29%] 2024-08-08T21:00:15.1803932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1807013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 29%] 2024-08-08T21:00:15.1810115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 29%] 2024-08-08T21:00:15.1813254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 29%] 2024-08-08T21:00:15.1816339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 29%] 2024-08-08T21:00:15.1819418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 29%] 2024-08-08T21:00:15.1822517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 29%] 2024-08-08T21:00:15.1825606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 29%] 2024-08-08T21:00:15.1828675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1831755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 29%] 2024-08-08T21:00:15.1834834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 29%] 2024-08-08T21:00:15.1846921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 29%] 2024-08-08T21:00:15.1850166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 29%] 2024-08-08T21:00:15.1853456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 29%] 2024-08-08T21:00:15.1856622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 29%] 2024-08-08T21:00:15.1860068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 29%] 2024-08-08T21:00:15.1863179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 29%] 2024-08-08T21:00:15.1866383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 29%] 2024-08-08T21:00:15.1869534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 29%] 2024-08-08T21:00:15.1872619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 29%] 2024-08-08T21:00:15.1875684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 29%] 2024-08-08T21:00:15.1878768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 29%] 2024-08-08T21:00:15.1881859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 30%] 2024-08-08T21:00:15.1884914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 30%] 2024-08-08T21:00:15.1888016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 30%] 2024-08-08T21:00:15.1891105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 30%] 2024-08-08T21:00:15.1894175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 30%] 2024-08-08T21:00:15.1897248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 30%] 2024-08-08T21:00:15.1900413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 30%] 2024-08-08T21:00:15.1903551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 30%] 2024-08-08T21:00:15.1906631Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 30%] 2024-08-08T21:00:15.1909705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 30%] 2024-08-08T21:00:15.1912855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 30%] 2024-08-08T21:00:15.1915984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 30%] 2024-08-08T21:00:15.1919057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 30%] 2024-08-08T21:00:15.1922138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 30%] 2024-08-08T21:00:15.1925196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 30%] 2024-08-08T21:00:15.1928268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 30%] 2024-08-08T21:00:15.1931343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 30%] 2024-08-08T21:00:15.1934432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 30%] 2024-08-08T21:00:15.1937522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 30%] 2024-08-08T21:00:15.1940575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 30%] 2024-08-08T21:00:15.1943706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 30%] 2024-08-08T21:00:15.1946826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 30%] 2024-08-08T21:00:15.1949892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 30%] 2024-08-08T21:00:15.1953001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 30%] 2024-08-08T21:00:15.1956107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 30%] 2024-08-08T21:00:15.1959494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 30%] 2024-08-08T21:00:15.1962583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 30%] 2024-08-08T21:00:15.1965662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0096s] [ 30%] 2024-08-08T21:00:15.1968767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0099s] [ 30%] 2024-08-08T21:00:15.1971870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 30%] 2024-08-08T21:00:15.1974935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 30%] 2024-08-08T21:00:15.1978035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0105s] [ 30%] 2024-08-08T21:00:15.1981149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0104s] [ 30%] 2024-08-08T21:00:15.1984263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0080s] [ 30%] 2024-08-08T21:00:15.1987363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 30%] 2024-08-08T21:00:15.1990546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0107s] [ 30%] 2024-08-08T21:00:15.1993707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 30%] 2024-08-08T21:00:15.1996798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0071s] [ 30%] 2024-08-08T21:00:15.1999883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 30%] 2024-08-08T21:00:15.2003054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0114s] [ 30%] 2024-08-08T21:00:15.2006225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 30%] 2024-08-08T21:00:15.2009336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 30%] 2024-08-08T21:00:15.2012430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 30%] 2024-08-08T21:00:15.2015545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0126s] [ 30%] 2024-08-08T21:00:15.2018634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0130s] [ 30%] 2024-08-08T21:00:15.2021717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0102s] [ 30%] 2024-08-08T21:00:15.2024811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0091s] [ 30%] 2024-08-08T21:00:15.2027909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0140s] [ 30%] 2024-08-08T21:00:15.2030996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0140s] [ 30%] 2024-08-08T21:00:15.2034143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0104s] [ 30%] 2024-08-08T21:00:15.2037301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0104s] [ 30%] 2024-08-08T21:00:15.2040398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0089s] [ 30%] 2024-08-08T21:00:15.2043475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0088s] [ 30%] 2024-08-08T21:00:15.2046584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 30%] 2024-08-08T21:00:15.2049718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 30%] 2024-08-08T21:00:15.2052840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0098s] [ 30%] 2024-08-08T21:00:15.2055936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0098s] [ 30%] 2024-08-08T21:00:15.2059275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 30%] 2024-08-08T21:00:15.2062393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 30%] 2024-08-08T21:00:15.2065452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 30%] 2024-08-08T21:00:15.2068541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 30%] 2024-08-08T21:00:15.2071625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 30%] 2024-08-08T21:00:15.2074709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 30%] 2024-08-08T21:00:15.2077801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 30%] 2024-08-08T21:00:15.2080979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 30%] 2024-08-08T21:00:15.2084135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 30%] 2024-08-08T21:00:15.2087225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 30%] 2024-08-08T21:00:15.2090302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 30%] 2024-08-08T21:00:15.2093452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 30%] 2024-08-08T21:00:15.2096614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 30%] 2024-08-08T21:00:15.2099688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 30%] 2024-08-08T21:00:15.2102758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 30%] 2024-08-08T21:00:15.2105875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 30%] 2024-08-08T21:00:15.2108959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 30%] 2024-08-08T21:00:15.2112038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 30%] 2024-08-08T21:00:15.2115118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 30%] 2024-08-08T21:00:15.2118199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 30%] 2024-08-08T21:00:15.2121263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 30%] 2024-08-08T21:00:15.2124488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 31%] 2024-08-08T21:00:15.2127533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 31%] 2024-08-08T21:00:15.2130726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 31%] 2024-08-08T21:00:15.2133713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 31%] 2024-08-08T21:00:15.2136829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 31%] 2024-08-08T21:00:15.2139962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 31%] 2024-08-08T21:00:15.2143033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 31%] 2024-08-08T21:00:15.2146108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 31%] 2024-08-08T21:00:15.2149186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 31%] 2024-08-08T21:00:15.2152318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 31%] 2024-08-08T21:00:15.2155383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 31%] 2024-08-08T21:00:15.2158800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 31%] 2024-08-08T21:00:15.2161859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 31%] 2024-08-08T21:00:15.2164941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 31%] 2024-08-08T21:00:15.2168027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 31%] 2024-08-08T21:00:15.2171183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 31%] 2024-08-08T21:00:15.2174317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 31%] 2024-08-08T21:00:15.2177509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 31%] 2024-08-08T21:00:15.2180492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 31%] 2024-08-08T21:00:15.2183623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 31%] 2024-08-08T21:00:15.2186785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 31%] 2024-08-08T21:00:15.2189871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 31%] 2024-08-08T21:00:15.2192923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 31%] 2024-08-08T21:00:15.2196006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 31%] 2024-08-08T21:00:15.2199073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 31%] 2024-08-08T21:00:15.2202146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 31%] 2024-08-08T21:00:15.2205230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 31%] 2024-08-08T21:00:15.2208299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 31%] 2024-08-08T21:00:15.2211380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 31%] 2024-08-08T21:00:15.2214438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 31%] 2024-08-08T21:00:15.2217570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 31%] 2024-08-08T21:00:15.2220697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 31%] 2024-08-08T21:00:15.2223764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 31%] 2024-08-08T21:00:15.2226826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 31%] 2024-08-08T21:00:15.2229957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 31%] 2024-08-08T21:00:15.2233080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 31%] 2024-08-08T21:00:15.2236144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 31%] 2024-08-08T21:00:15.2239228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 31%] 2024-08-08T21:00:15.2242300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 31%] 2024-08-08T21:00:15.2245346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 31%] 2024-08-08T21:00:15.2248405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 31%] 2024-08-08T21:00:15.2251468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 31%] 2024-08-08T21:00:15.2254603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 31%] 2024-08-08T21:00:15.2257673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 31%] 2024-08-08T21:00:15.2261108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 31%] 2024-08-08T21:00:15.2264262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0192s] [ 31%] 2024-08-08T21:00:15.2267391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0202s] [ 31%] 2024-08-08T21:00:15.2270505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0196s] [ 31%] 2024-08-08T21:00:15.2273750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0193s] [ 31%] 2024-08-08T21:00:15.2276951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0297s] [ 31%] 2024-08-08T21:00:15.2280059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0305s] [ 31%] 2024-08-08T21:00:15.2283189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0253s] [ 31%] 2024-08-08T21:00:15.2286329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0262s] [ 31%] 2024-08-08T21:00:15.2289447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0212s] [ 31%] 2024-08-08T21:00:15.2292562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0219s] [ 31%] 2024-08-08T21:00:15.2295676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0219s] [ 31%] 2024-08-08T21:00:15.2298787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0216s] [ 31%] 2024-08-08T21:00:15.2301902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0310s] [ 31%] 2024-08-08T21:00:15.2305023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0315s] [ 31%] 2024-08-08T21:00:15.2308199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0272s] [ 31%] 2024-08-08T21:00:15.2311378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1971s] [ 31%] 2024-08-08T21:00:15.2314485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0241s] [ 31%] 2024-08-08T21:00:15.2317580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0264s] [ 31%] 2024-08-08T21:00:15.2320745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0266s] [ 31%] 2024-08-08T21:00:15.2323900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0261s] [ 31%] 2024-08-08T21:00:15.2327017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0346s] [ 31%] 2024-08-08T21:00:15.2330147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0349s] [ 31%] 2024-08-08T21:00:15.2333282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0362s] [ 31%] 2024-08-08T21:00:15.2336387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0362s] [ 31%] 2024-08-08T21:00:15.2339502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0188s] [ 31%] 2024-08-08T21:00:15.2342613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0192s] [ 31%] 2024-08-08T21:00:15.2345722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0188s] [ 31%] 2024-08-08T21:00:15.2348834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0188s] [ 31%] 2024-08-08T21:00:15.2351995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0294s] [ 31%] 2024-08-08T21:00:15.2355193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0301s] [ 31%] 2024-08-08T21:00:15.2358310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0249s] [ 31%] 2024-08-08T21:00:15.2361662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0248s] [ 31%] 2024-08-08T21:00:15.2364865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0352s] [ 31%] 2024-08-08T21:00:15.2368062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0381s] [ 32%] 2024-08-08T21:00:15.2371176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0345s] [ 32%] 2024-08-08T21:00:15.2374272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.1973s] [ 32%] 2024-08-08T21:00:15.2377429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0567s] [ 32%] 2024-08-08T21:00:15.2380533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0581s] [ 32%] 2024-08-08T21:00:15.2383654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0471s] [ 32%] 2024-08-08T21:00:15.2386780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0473s] [ 32%] 2024-08-08T21:00:15.2389902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0372s] [ 32%] 2024-08-08T21:00:15.2392989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0406s] [ 32%] 2024-08-08T21:00:15.2396104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0393s] [ 32%] 2024-08-08T21:00:15.2399280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0394s] [ 32%] 2024-08-08T21:00:15.2402474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0577s] [ 32%] 2024-08-08T21:00:15.2405601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0597s] [ 32%] 2024-08-08T21:00:15.2408756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0512s] [ 32%] 2024-08-08T21:00:15.2411936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.2124s] [ 32%] 2024-08-08T21:00:15.2415057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0424s] [ 32%] 2024-08-08T21:00:15.2418165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0468s] [ 32%] 2024-08-08T21:00:15.2421289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0461s] [ 32%] 2024-08-08T21:00:15.2424415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0462s] [ 32%] 2024-08-08T21:00:15.2427508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0632s] [ 32%] 2024-08-08T21:00:15.2430642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0641s] [ 32%] 2024-08-08T21:00:15.2433776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0670s] [ 32%] 2024-08-08T21:00:15.2436909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0679s] [ 32%] 2024-08-08T21:00:15.2440018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0337s] [ 32%] 2024-08-08T21:00:15.2443174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0356s] [ 32%] 2024-08-08T21:00:15.2446336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.1948s] [ 32%] 2024-08-08T21:00:15.2449437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0332s] [ 32%] 2024-08-08T21:00:15.2452594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0567s] [ 32%] 2024-08-08T21:00:15.2455745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0579s] [ 32%] 2024-08-08T21:00:15.2459188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0443s] [ 32%] 2024-08-08T21:00:15.2462326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0448s] [ 32%] 2024-08-08T21:00:15.2465422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0114s] [ 32%] 2024-08-08T21:00:15.2468547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 32%] 2024-08-08T21:00:15.2471651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0111s] [ 32%] 2024-08-08T21:00:15.2474786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0114s] [ 32%] 2024-08-08T21:00:15.2477877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0167s] [ 32%] 2024-08-08T21:00:15.2481006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0169s] [ 32%] 2024-08-08T21:00:15.2484143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0138s] [ 32%] 2024-08-08T21:00:15.2487305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0140s] [ 32%] 2024-08-08T21:00:15.2490491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0126s] [ 32%] 2024-08-08T21:00:15.2493591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0127s] [ 32%] 2024-08-08T21:00:15.2496706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0128s] [ 32%] 2024-08-08T21:00:15.2499885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0127s] [ 32%] 2024-08-08T21:00:15.2503055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0177s] [ 32%] 2024-08-08T21:00:15.2506167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0179s] [ 32%] 2024-08-08T21:00:15.2509271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0152s] [ 32%] 2024-08-08T21:00:15.2512382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0151s] [ 32%] 2024-08-08T21:00:15.2515510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0153s] [ 32%] 2024-08-08T21:00:15.2518616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0155s] [ 32%] 2024-08-08T21:00:15.2521707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0157s] [ 32%] 2024-08-08T21:00:15.2524815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0159s] [ 32%] 2024-08-08T21:00:15.2527916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0202s] [ 32%] 2024-08-08T21:00:15.2531027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0197s] [ 32%] 2024-08-08T21:00:15.2534191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.1836s] [ 32%] 2024-08-08T21:00:15.2537338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0189s] [ 32%] 2024-08-08T21:00:15.2540439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0109s] [ 32%] 2024-08-08T21:00:15.2543537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0112s] [ 32%] 2024-08-08T21:00:15.2546672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0108s] [ 32%] 2024-08-08T21:00:15.2549809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0107s] [ 32%] 2024-08-08T21:00:15.2552943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0161s] [ 32%] 2024-08-08T21:00:15.2556037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0163s] [ 32%] 2024-08-08T21:00:15.2559547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0131s] [ 32%] 2024-08-08T21:00:15.2562673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0131s] [ 32%] 2024-08-08T21:00:15.2565773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0106s] [ 32%] 2024-08-08T21:00:15.2568872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 32%] 2024-08-08T21:00:15.2571983Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0096s] [ 32%] 2024-08-08T21:00:15.2575047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 32%] 2024-08-08T21:00:15.2578242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0145s] [ 32%] 2024-08-08T21:00:15.2581429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0144s] [ 32%] 2024-08-08T21:00:15.2584536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0109s] [ 32%] 2024-08-08T21:00:15.2587944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0106s] [ 32%] 2024-08-08T21:00:15.2591209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0111s] [ 32%] 2024-08-08T21:00:15.2594379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0112s] [ 32%] 2024-08-08T21:00:15.2597545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0102s] [ 32%] 2024-08-08T21:00:15.2600548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0106s] [ 32%] 2024-08-08T21:00:15.2603651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0148s] [ 32%] 2024-08-08T21:00:15.2606760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0150s] [ 32%] 2024-08-08T21:00:15.2609862Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0111s] [ 32%] 2024-08-08T21:00:15.2612956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 33%] 2024-08-08T21:00:15.2616065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0136s] [ 33%] 2024-08-08T21:00:15.2619160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0134s] [ 33%] 2024-08-08T21:00:15.2622243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0132s] [ 33%] 2024-08-08T21:00:15.2625406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0132s] [ 33%] 2024-08-08T21:00:15.2628541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0168s] [ 33%] 2024-08-08T21:00:15.2631643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0169s] [ 33%] 2024-08-08T21:00:15.2634748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0137s] [ 33%] 2024-08-08T21:00:15.2637914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0136s] [ 33%] 2024-08-08T21:00:15.2641052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0093s] [ 33%] 2024-08-08T21:00:15.2644127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 33%] 2024-08-08T21:00:15.2647191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 33%] 2024-08-08T21:00:15.2650282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 33%] 2024-08-08T21:00:15.2653401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0130s] [ 33%] 2024-08-08T21:00:15.2656496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0129s] [ 33%] 2024-08-08T21:00:15.2659829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0092s] [ 33%] 2024-08-08T21:00:15.2662941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 33%] 2024-08-08T21:00:15.2666024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0675s] [ 33%] 2024-08-08T21:00:15.2669232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0733s] [ 33%] 2024-08-08T21:00:15.2672428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0639s] [ 33%] 2024-08-08T21:00:15.2675543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.2144s] [ 33%] 2024-08-08T21:00:15.2678663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1097s] [ 33%] 2024-08-08T21:00:15.2681858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1135s] [ 33%] 2024-08-08T21:00:15.2685049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0910s] [ 33%] 2024-08-08T21:00:15.2688175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0892s] [ 33%] 2024-08-08T21:00:15.2691286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0718s] [ 33%] 2024-08-08T21:00:15.2694422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0785s] [ 33%] 2024-08-08T21:00:15.2697548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.2298s] [ 33%] 2024-08-08T21:00:15.2700658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0719s] [ 33%] 2024-08-08T21:00:15.2703769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1128s] [ 33%] 2024-08-08T21:00:15.2706908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1161s] [ 33%] 2024-08-08T21:00:15.2710033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0955s] [ 33%] 2024-08-08T21:00:15.2713160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.2374s] [ 33%] 2024-08-08T21:00:15.2716330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0811s] [ 33%] 2024-08-08T21:00:15.2719494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0884s] [ 33%] 2024-08-08T21:00:15.2722591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0846s] [ 33%] 2024-08-08T21:00:15.2725706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0863s] [ 33%] 2024-08-08T21:00:15.2728873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1216s] [ 33%] 2024-08-08T21:00:15.2732069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.2817s] [ 33%] 2024-08-08T21:00:15.2735189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.1262s] [ 33%] 2024-08-08T21:00:15.2738279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1264s] [ 33%] 2024-08-08T21:00:15.2741401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0653s] [ 33%] 2024-08-08T21:00:15.2744498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0699s] [ 33%] 2024-08-08T21:00:15.2747603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0605s] [ 33%] 2024-08-08T21:00:15.2750714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.2099s] [ 33%] 2024-08-08T21:00:15.2753877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1082s] [ 33%] 2024-08-08T21:00:15.2756982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1112s] [ 33%] 2024-08-08T21:00:15.2760471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0862s] [ 33%] 2024-08-08T21:00:15.2763674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0859s] [ 33%] 2024-08-08T21:00:15.2766779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0125s] [ 33%] 2024-08-08T21:00:15.2769886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0127s] [ 33%] 2024-08-08T21:00:15.2773050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0124s] [ 33%] 2024-08-08T21:00:15.2776220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0125s] [ 33%] 2024-08-08T21:00:15.2779327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0181s] [ 33%] 2024-08-08T21:00:15.2782445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0178s] [ 33%] 2024-08-08T21:00:15.2785568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0158s] [ 33%] 2024-08-08T21:00:15.2788701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0156s] [ 33%] 2024-08-08T21:00:15.2791806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0131s] [ 33%] 2024-08-08T21:00:15.2794890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0136s] [ 33%] 2024-08-08T21:00:15.2798002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.1788s] [ 33%] 2024-08-08T21:00:15.2801113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0143s] [ 33%] 2024-08-08T21:00:15.2804202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0188s] [ 33%] 2024-08-08T21:00:15.2807375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0184s] [ 33%] 2024-08-08T21:00:15.2810545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0169s] [ 33%] 2024-08-08T21:00:15.2813637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0170s] [ 33%] 2024-08-08T21:00:15.2816792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0151s] [ 33%] 2024-08-08T21:00:15.2819947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0161s] [ 33%] 2024-08-08T21:00:15.2823052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0171s] [ 33%] 2024-08-08T21:00:15.2826150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0168s] [ 33%] 2024-08-08T21:00:15.2829264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0204s] [ 33%] 2024-08-08T21:00:15.2832378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0206s] [ 33%] 2024-08-08T21:00:15.2835492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0217s] [ 33%] 2024-08-08T21:00:15.2838604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0223s] [ 33%] 2024-08-08T21:00:15.2841716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0120s] [ 33%] 2024-08-08T21:00:15.2844812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0122s] [ 33%] 2024-08-08T21:00:15.2847876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0122s] [ 33%] 2024-08-08T21:00:15.2851013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0123s] [ 33%] 2024-08-08T21:00:15.2854224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0174s] [ 33%] 2024-08-08T21:00:15.2857321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0171s] [ 34%] 2024-08-08T21:00:15.2860710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0150s] [ 34%] 2024-08-08T21:00:15.2863908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0150s] [ 34%] 2024-08-08T21:00:15.2867067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0105s] [ 34%] 2024-08-08T21:00:15.2870152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0107s] [ 34%] 2024-08-08T21:00:15.2873243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0097s] [ 34%] 2024-08-08T21:00:15.2874856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0100s] [ 34%] 2024-08-08T21:00:15.2876447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0150s] [ 34%] 2024-08-08T21:00:15.2878037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0148s] [ 34%] 2024-08-08T21:00:15.2879649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0111s] [ 34%] 2024-08-08T21:00:15.2881256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0108s] [ 34%] 2024-08-08T21:00:15.2882843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0113s] [ 34%] 2024-08-08T21:00:15.2884432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 34%] 2024-08-08T21:00:15.2886085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0103s] [ 34%] 2024-08-08T21:00:15.2887744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 34%] 2024-08-08T21:00:15.2889341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0146s] [ 34%] 2024-08-08T21:00:15.2891211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0145s] [ 34%] 2024-08-08T21:00:15.2892883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0107s] [ 34%] 2024-08-08T21:00:15.2894481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0110s] [ 34%] 2024-08-08T21:00:15.2896054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0132s] [ 34%] 2024-08-08T21:00:15.2897660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0132s] [ 34%] 2024-08-08T21:00:15.2899237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0127s] [ 34%] 2024-08-08T21:00:15.2900851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0129s] [ 34%] 2024-08-08T21:00:15.2902423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0170s] [ 34%] 2024-08-08T21:00:15.2904037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0169s] [ 34%] 2024-08-08T21:00:15.2905622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0140s] [ 34%] 2024-08-08T21:00:15.2907232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0136s] [ 34%] 2024-08-08T21:00:15.2908848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0093s] [ 34%] 2024-08-08T21:00:15.2910498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 34%] 2024-08-08T21:00:15.2912060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 34%] 2024-08-08T21:00:15.2913640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 34%] 2024-08-08T21:00:15.2915294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0133s] [ 34%] 2024-08-08T21:00:15.2916930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0131s] [ 34%] 2024-08-08T21:00:15.2918516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0091s] [ 34%] 2024-08-08T21:00:15.2920099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0091s] [ 34%] 2024-08-08T21:00:15.2921705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 34%] 2024-08-08T21:00:15.2923302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 34%] 2024-08-08T21:00:15.2924900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 34%] 2024-08-08T21:00:15.2926507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 34%] 2024-08-08T21:00:15.2928117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 34%] 2024-08-08T21:00:15.2929722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 34%] 2024-08-08T21:00:15.2931333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 34%] 2024-08-08T21:00:15.2932981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 34%] 2024-08-08T21:00:15.2934669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 34%] 2024-08-08T21:00:15.2936274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 34%] 2024-08-08T21:00:15.2937903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 34%] 2024-08-08T21:00:15.2939568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 34%] 2024-08-08T21:00:15.2941154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0083s] [ 34%] 2024-08-08T21:00:15.2942767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 34%] 2024-08-08T21:00:15.2944355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 34%] 2024-08-08T21:00:15.2945980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 34%] 2024-08-08T21:00:15.2947561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 34%] 2024-08-08T21:00:15.2949175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 34%] 2024-08-08T21:00:15.2950772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0082s] [ 34%] 2024-08-08T21:00:15.2952374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 34%] 2024-08-08T21:00:15.2954019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0096s] [ 34%] 2024-08-08T21:00:15.2955667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 34%] 2024-08-08T21:00:15.2957329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0087s] [ 34%] 2024-08-08T21:00:15.2959155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 34%] 2024-08-08T21:00:15.2960776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 34%] 2024-08-08T21:00:15.2962423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 34%] 2024-08-08T21:00:15.2964088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 34%] 2024-08-08T21:00:15.2965668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 34%] 2024-08-08T21:00:15.2967272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0080s] [ 34%] 2024-08-08T21:00:15.2968873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 34%] 2024-08-08T21:00:15.2970460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 34%] 2024-08-08T21:00:15.2972075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 34%] 2024-08-08T21:00:15.2973659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0082s] [ 34%] 2024-08-08T21:00:15.2975281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0090s] [ 34%] 2024-08-08T21:00:15.2976869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0078s] [ 34%] 2024-08-08T21:00:15.2978537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 34%] 2024-08-08T21:00:15.2980195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0108s] [ 34%] 2024-08-08T21:00:15.2981817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0110s] [ 34%] 2024-08-08T21:00:15.2983408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0097s] [ 34%] 2024-08-08T21:00:15.2985069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0097s] [ 34%] 2024-08-08T21:00:15.2986707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0086s] [ 34%] 2024-08-08T21:00:15.2988307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 34%] 2024-08-08T21:00:15.2989910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0090s] [ 34%] 2024-08-08T21:00:15.2991504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 35%] 2024-08-08T21:00:15.2993112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0115s] [ 35%] 2024-08-08T21:00:15.2994716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0115s] [ 35%] 2024-08-08T21:00:15.2996323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0107s] [ 35%] 2024-08-08T21:00:15.2997922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0103s] [ 35%] 2024-08-08T21:00:15.2999523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0097s] [ 35%] 2024-08-08T21:00:15.3001119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 35%] 2024-08-08T21:00:15.3002769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0102s] [ 35%] 2024-08-08T21:00:15.3004433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 35%] 2024-08-08T21:00:15.3006021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0127s] [ 35%] 2024-08-08T21:00:15.3007637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0129s] [ 35%] 2024-08-08T21:00:15.3009264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0132s] [ 35%] 2024-08-08T21:00:15.3010928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0135s] [ 35%] 2024-08-08T21:00:15.3012508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0081s] [ 35%] 2024-08-08T21:00:15.3014116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 35%] 2024-08-08T21:00:15.3015692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0079s] [ 35%] 2024-08-08T21:00:15.3017290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 35%] 2024-08-08T21:00:15.3018865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0106s] [ 35%] 2024-08-08T21:00:15.3020476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0107s] [ 35%] 2024-08-08T21:00:15.3022072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0095s] [ 35%] 2024-08-08T21:00:15.3023669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 35%] 2024-08-08T21:00:15.3025309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 35%] 2024-08-08T21:00:15.3026949Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3028542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3030120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3031760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3033402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3035004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 35%] 2024-08-08T21:00:15.3036598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3038202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3039794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 35%] 2024-08-08T21:00:15.3041364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3042962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3044548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 35%] 2024-08-08T21:00:15.3046155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3047737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 35%] 2024-08-08T21:00:15.3049392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 35%] 2024-08-08T21:00:15.3051021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3052659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3054230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 35%] 2024-08-08T21:00:15.3055874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 35%] 2024-08-08T21:00:15.3057502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 35%] 2024-08-08T21:00:15.3059363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 35%] 2024-08-08T21:00:15.3060976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 35%] 2024-08-08T21:00:15.3062583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 35%] 2024-08-08T21:00:15.3064171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 35%] 2024-08-08T21:00:15.3065752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3067341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3068924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3070516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3072177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3073833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 35%] 2024-08-08T21:00:15.3075423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 35%] 2024-08-08T21:00:15.3077012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3078663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 35%] 2024-08-08T21:00:15.3080301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 35%] 2024-08-08T21:00:15.3081889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3083455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3085059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3086634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 35%] 2024-08-08T21:00:15.3088237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 35%] 2024-08-08T21:00:15.3089803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3091401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3092962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3094552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3096177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3097810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3099394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 35%] 2024-08-08T21:00:15.3100978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 35%] 2024-08-08T21:00:15.3102604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3104234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 35%] 2024-08-08T21:00:15.3105809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3107386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 35%] 2024-08-08T21:00:15.3108973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 35%] 2024-08-08T21:00:15.3110555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 35%] 2024-08-08T21:00:15.3112141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 35%] 2024-08-08T21:00:15.3113751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 35%] 2024-08-08T21:00:15.3115299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 36%] 2024-08-08T21:00:15.3116896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 36%] 2024-08-08T21:00:15.3118464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 36%] 2024-08-08T21:00:15.3120101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 36%] 2024-08-08T21:00:15.3121726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 36%] 2024-08-08T21:00:15.3123323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 36%] 2024-08-08T21:00:15.3124952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 36%] 2024-08-08T21:00:15.3126589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 36%] 2024-08-08T21:00:15.3128170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0124s] [ 36%] 2024-08-08T21:00:15.3129782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0132s] [ 36%] 2024-08-08T21:00:15.3131377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0113s] [ 36%] 2024-08-08T21:00:15.3132978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0113s] [ 36%] 2024-08-08T21:00:15.3134588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0173s] [ 36%] 2024-08-08T21:00:15.3136193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0179s] [ 36%] 2024-08-08T21:00:15.3137813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0143s] [ 36%] 2024-08-08T21:00:15.3139417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0143s] [ 36%] 2024-08-08T21:00:15.3141018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0132s] [ 36%] 2024-08-08T21:00:15.3142654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0141s] [ 36%] 2024-08-08T21:00:15.3144310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0129s] [ 36%] 2024-08-08T21:00:15.3145906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0129s] [ 36%] 2024-08-08T21:00:15.3147511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0180s] [ 36%] 2024-08-08T21:00:15.3149151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0184s] [ 36%] 2024-08-08T21:00:15.3150799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0158s] [ 36%] 2024-08-08T21:00:15.3152415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0157s] [ 36%] 2024-08-08T21:00:15.3154051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0152s] [ 36%] 2024-08-08T21:00:15.3155673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0162s] [ 36%] 2024-08-08T21:00:15.3157259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0154s] [ 36%] 2024-08-08T21:00:15.3159150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0154s] [ 36%] 2024-08-08T21:00:15.3160705Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0201s] [ 36%] 2024-08-08T21:00:15.3162336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0204s] [ 36%] 2024-08-08T21:00:15.3163929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0206s] [ 36%] 2024-08-08T21:00:15.3165618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0206s] [ 36%] 2024-08-08T21:00:15.3167265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0124s] [ 36%] 2024-08-08T21:00:15.3168861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0129s] [ 36%] 2024-08-08T21:00:15.3170447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0122s] [ 36%] 2024-08-08T21:00:15.3172086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0111s] [ 36%] 2024-08-08T21:00:15.3173765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0172s] [ 36%] 2024-08-08T21:00:15.3175360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0177s] [ 36%] 2024-08-08T21:00:15.3176960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0141s] [ 36%] 2024-08-08T21:00:15.3178564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0142s] [ 36%] 2024-08-08T21:00:15.3180164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 36%] 2024-08-08T21:00:15.3181754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 36%] 2024-08-08T21:00:15.3183347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 36%] 2024-08-08T21:00:15.3184936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 36%] 2024-08-08T21:00:15.3186526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 36%] 2024-08-08T21:00:15.3188141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 36%] 2024-08-08T21:00:15.3189778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 36%] 2024-08-08T21:00:15.3191446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 36%] 2024-08-08T21:00:15.3193020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 36%] 2024-08-08T21:00:15.3194617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 36%] 2024-08-08T21:00:15.3196227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 36%] 2024-08-08T21:00:15.3197894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 36%] 2024-08-08T21:00:15.3199453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 36%] 2024-08-08T21:00:15.3201072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 36%] 2024-08-08T21:00:15.3202666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 36%] 2024-08-08T21:00:15.3204264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 36%] 2024-08-08T21:00:15.3205854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 36%] 2024-08-08T21:00:15.3207444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 36%] 2024-08-08T21:00:15.3209034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 36%] 2024-08-08T21:00:15.3210614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 36%] 2024-08-08T21:00:15.3212252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 36%] 2024-08-08T21:00:15.3213898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 36%] 2024-08-08T21:00:15.3215505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 36%] 2024-08-08T21:00:15.3217101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 36%] 2024-08-08T21:00:15.3218730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 36%] 2024-08-08T21:00:15.3220365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 36%] 2024-08-08T21:00:15.3221931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 36%] 2024-08-08T21:00:15.3223525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 36%] 2024-08-08T21:00:15.3225107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 36%] 2024-08-08T21:00:15.3226718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 36%] 2024-08-08T21:00:15.3228283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 36%] 2024-08-08T21:00:15.3229882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 36%] 2024-08-08T21:00:15.3231457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 36%] 2024-08-08T21:00:15.3233055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 36%] 2024-08-08T21:00:15.3234614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 36%] 2024-08-08T21:00:15.3236248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 36%] 2024-08-08T21:00:15.3237882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 36%] 2024-08-08T21:00:15.3239464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 36%] 2024-08-08T21:00:15.3241049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 37%] 2024-08-08T21:00:15.3242670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3244301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 37%] 2024-08-08T21:00:15.3245878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 37%] 2024-08-08T21:00:15.3247460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 37%] 2024-08-08T21:00:15.3249040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 37%] 2024-08-08T21:00:15.3250634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3252215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 37%] 2024-08-08T21:00:15.3253866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 37%] 2024-08-08T21:00:15.3255431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 37%] 2024-08-08T21:00:15.3256997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 37%] 2024-08-08T21:00:15.3258901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 37%] 2024-08-08T21:00:15.3260457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 37%] 2024-08-08T21:00:15.3262130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 37%] 2024-08-08T21:00:15.3263702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3265366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3267762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3269356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3270918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 37%] 2024-08-08T21:00:15.3272522Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 37%] 2024-08-08T21:00:15.3274097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 37%] 2024-08-08T21:00:15.3275693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 37%] 2024-08-08T21:00:15.3277268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 37%] 2024-08-08T21:00:15.3278853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 37%] 2024-08-08T21:00:15.3280438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3282011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3283659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 37%] 2024-08-08T21:00:15.3285307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 37%] 2024-08-08T21:00:15.3286897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 37%] 2024-08-08T21:00:15.3288483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 37%] 2024-08-08T21:00:15.3290122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 37%] 2024-08-08T21:00:15.3291764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3293372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3294976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 37%] 2024-08-08T21:00:15.3296558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 37%] 2024-08-08T21:00:15.3298165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 37%] 2024-08-08T21:00:15.3299732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 37%] 2024-08-08T21:00:15.3301343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 37%] 2024-08-08T21:00:15.3302932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3304542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3306125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3307831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3309461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 37%] 2024-08-08T21:00:15.3311068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 37%] 2024-08-08T21:00:15.3312645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 37%] 2024-08-08T21:00:15.3314276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 37%] 2024-08-08T21:00:15.3315927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3317524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3319130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3320739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3322325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 37%] 2024-08-08T21:00:15.3323905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 37%] 2024-08-08T21:00:15.3325494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 37%] 2024-08-08T21:00:15.3327077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 37%] 2024-08-08T21:00:15.3328669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3330306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 37%] 2024-08-08T21:00:15.3331930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 37%] 2024-08-08T21:00:15.3333528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 37%] 2024-08-08T21:00:15.3335116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3336738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3338362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 37%] 2024-08-08T21:00:15.3339950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 37%] 2024-08-08T21:00:15.3341526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 37%] 2024-08-08T21:00:15.3343140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 37%] 2024-08-08T21:00:15.3344731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 37%] 2024-08-08T21:00:15.3346341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 37%] 2024-08-08T21:00:15.3347924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0070s] [ 37%] 2024-08-08T21:00:15.3349521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 37%] 2024-08-08T21:00:15.3351107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 37%] 2024-08-08T21:00:15.3352733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 37%] 2024-08-08T21:00:15.3354376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 37%] 2024-08-08T21:00:15.3356029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 37%] 2024-08-08T21:00:15.3357654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 37%] 2024-08-08T21:00:15.3359494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 37%] 2024-08-08T21:00:15.3361188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0083s] [ 37%] 2024-08-08T21:00:15.3362845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 37%] 2024-08-08T21:00:15.3364432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3366015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 37%] 2024-08-08T21:00:15.3367605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0089s] [ 38%] 2024-08-08T21:00:15.3369215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0090s] [ 38%] 2024-08-08T21:00:15.3370797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0077s] [ 38%] 2024-08-08T21:00:15.3372418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 38%] 2024-08-08T21:00:15.3373996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 38%] 2024-08-08T21:00:15.3375594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 38%] 2024-08-08T21:00:15.3377217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 38%] 2024-08-08T21:00:15.3378879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 38%] 2024-08-08T21:00:15.3380457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 38%] 2024-08-08T21:00:15.3382055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 38%] 2024-08-08T21:00:15.3383666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 38%] 2024-08-08T21:00:15.3385294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3386880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3388464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3390054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 38%] 2024-08-08T21:00:15.3391628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 38%] 2024-08-08T21:00:15.3393209Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 38%] 2024-08-08T21:00:15.3394795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3396384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 38%] 2024-08-08T21:00:15.3397960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3399540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 38%] 2024-08-08T21:00:15.3401162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3402774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 38%] 2024-08-08T21:00:15.3404362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 38%] 2024-08-08T21:00:15.3405933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3407570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3409180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 38%] 2024-08-08T21:00:15.3410770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3412340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3413940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3415497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 38%] 2024-08-08T21:00:15.3417082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 38%] 2024-08-08T21:00:15.3418654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3420236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3421824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3423439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3425069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3426652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3428237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 38%] 2024-08-08T21:00:15.3429853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 38%] 2024-08-08T21:00:15.3431499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3433070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3434650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3436230Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_37_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 38%] 2024-08-08T21:00:15.3437807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3439411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 38%] 2024-08-08T21:00:15.3441132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 38%] 2024-08-08T21:00:15.3442738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 38%] 2024-08-08T21:00:15.3444306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3445902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 38%] 2024-08-08T21:00:15.3447542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3449194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3450755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3452350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3454002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 38%] 2024-08-08T21:00:15.3455632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 38%] 2024-08-08T21:00:15.3457219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3459054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3460667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3462248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3463824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 38%] 2024-08-08T21:00:15.3465411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 38%] 2024-08-08T21:00:15.3467008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 38%] 2024-08-08T21:00:15.3468581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 38%] 2024-08-08T21:00:15.3470252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3471906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 38%] 2024-08-08T21:00:15.3473479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3475080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3476706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 38%] 2024-08-08T21:00:15.3478373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3479938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 38%] 2024-08-08T21:00:15.3481520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 38%] 2024-08-08T21:00:15.3483088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 38%] 2024-08-08T21:00:15.3484695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 38%] 2024-08-08T21:00:15.3486263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 38%] 2024-08-08T21:00:15.3487867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 38%] 2024-08-08T21:00:15.3489457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0097s] [ 38%] 2024-08-08T21:00:15.3491051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 38%] 2024-08-08T21:00:15.3492641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 39%] 2024-08-08T21:00:15.3494301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 39%] 2024-08-08T21:00:15.3495922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0114s] [ 39%] 2024-08-08T21:00:15.3497518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0107s] [ 39%] 2024-08-08T21:00:15.3499122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0083s] [ 39%] 2024-08-08T21:00:15.3500785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 39%] 2024-08-08T21:00:15.3502428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0106s] [ 39%] 2024-08-08T21:00:15.3504020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0109s] [ 39%] 2024-08-08T21:00:15.3505614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0084s] [ 39%] 2024-08-08T21:00:15.3507208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 39%] 2024-08-08T21:00:15.3508796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0121s] [ 39%] 2024-08-08T21:00:15.3510409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0122s] [ 39%] 2024-08-08T21:00:15.3512000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 39%] 2024-08-08T21:00:15.3513628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 39%] 2024-08-08T21:00:15.3515208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0132s] [ 39%] 2024-08-08T21:00:15.3516801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0135s] [ 39%] 2024-08-08T21:00:15.3518420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0099s] [ 39%] 2024-08-08T21:00:15.3520074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0102s] [ 39%] 2024-08-08T21:00:15.3521650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0146s] [ 39%] 2024-08-08T21:00:15.3523304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0146s] [ 39%] 2024-08-08T21:00:15.3524962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0115s] [ 39%] 2024-08-08T21:00:15.3526567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0114s] [ 39%] 2024-08-08T21:00:15.3528152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 39%] 2024-08-08T21:00:15.3529740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0093s] [ 39%] 2024-08-08T21:00:15.3531324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 39%] 2024-08-08T21:00:15.3532899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 39%] 2024-08-08T21:00:15.3534515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0102s] [ 39%] 2024-08-08T21:00:15.3536086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 39%] 2024-08-08T21:00:15.3537685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0089s] [ 39%] 2024-08-08T21:00:15.3539262Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 39%] 2024-08-08T21:00:15.3540894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 39%] 2024-08-08T21:00:15.3542529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 39%] 2024-08-08T21:00:15.3544096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 39%] 2024-08-08T21:00:15.3545693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 39%] 2024-08-08T21:00:15.3547307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 39%] 2024-08-08T21:00:15.3548954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 39%] 2024-08-08T21:00:15.3550529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 39%] 2024-08-08T21:00:15.3552129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 39%] 2024-08-08T21:00:15.3553752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 39%] 2024-08-08T21:00:15.3555355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 39%] 2024-08-08T21:00:15.3556908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 39%] 2024-08-08T21:00:15.3558754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 39%] 2024-08-08T21:00:15.3560321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 39%] 2024-08-08T21:00:15.3561909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 39%] 2024-08-08T21:00:15.3563499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 39%] 2024-08-08T21:00:15.3565153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 39%] 2024-08-08T21:00:15.3566807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 39%] 2024-08-08T21:00:15.3568389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 39%] 2024-08-08T21:00:15.3569972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 39%] 2024-08-08T21:00:15.3571640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 39%] 2024-08-08T21:00:15.3573303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 39%] 2024-08-08T21:00:15.3574894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 39%] 2024-08-08T21:00:15.3576490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 39%] 2024-08-08T21:00:15.3578079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 39%] 2024-08-08T21:00:15.3579647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 39%] 2024-08-08T21:00:15.3581243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 39%] 2024-08-08T21:00:15.3582819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 39%] 2024-08-08T21:00:15.3584422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 39%] 2024-08-08T21:00:15.3585996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 39%] 2024-08-08T21:00:15.3587644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 39%] 2024-08-08T21:00:15.3589267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 39%] 2024-08-08T21:00:15.3590864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 39%] 2024-08-08T21:00:15.3592432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 39%] 2024-08-08T21:00:15.3594075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 39%] 2024-08-08T21:00:15.3595696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 39%] 2024-08-08T21:00:15.3597270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 39%] 2024-08-08T21:00:15.3598861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 39%] 2024-08-08T21:00:15.3600452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 39%] 2024-08-08T21:00:15.3602043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 39%] 2024-08-08T21:00:15.3603615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 39%] 2024-08-08T21:00:15.3605198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 39%] 2024-08-08T21:00:15.3606785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 39%] 2024-08-08T21:00:15.3608368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 39%] 2024-08-08T21:00:15.3609943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 39%] 2024-08-08T21:00:15.3611600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 39%] 2024-08-08T21:00:15.3613242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 39%] 2024-08-08T21:00:15.3614805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 39%] 2024-08-08T21:00:15.3616401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 39%] 2024-08-08T21:00:15.3618028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 40%] 2024-08-08T21:00:15.3619679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 40%] 2024-08-08T21:00:15.3621273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 40%] 2024-08-08T21:00:15.3622848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 40%] 2024-08-08T21:00:15.3624436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 40%] 2024-08-08T21:00:15.3626034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 40%] 2024-08-08T21:00:15.3630612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 40%] 2024-08-08T21:00:15.3634852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 40%] 2024-08-08T21:00:15.3636454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 40%] 2024-08-08T21:00:15.3638042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 40%] 2024-08-08T21:00:15.3639670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 40%] 2024-08-08T21:00:15.3641293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 40%] 2024-08-08T21:00:15.3642874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 40%] 2024-08-08T21:00:15.3644466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 40%] 2024-08-08T21:00:15.3646114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 40%] 2024-08-08T21:00:15.3647770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 40%] 2024-08-08T21:00:15.3648721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_mask_variants_mask_dim_1_cuda PASSED [0.0038s] [ 40%] 2024-08-08T21:00:15.3649667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_mask_variants_mask_dim_2_cuda PASSED [0.0019s] [ 40%] 2024-08-08T21:00:15.3650622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_mask_variants_mask_dim_3_cuda PASSED [0.0019s] [ 40%] 2024-08-08T21:00:15.3651565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_mask_variants_mask_dim_4_cuda PASSED [0.0019s] [ 40%] 2024-08-08T21:00:15.3653194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0071s] [ 40%] 2024-08-08T21:00:15.3654745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 40%] 2024-08-08T21:00:15.3656291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 40%] 2024-08-08T21:00:15.3657932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 40%] 2024-08-08T21:00:15.3659858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0095s] [ 40%] 2024-08-08T21:00:15.3661446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 40%] 2024-08-08T21:00:15.3662998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 40%] 2024-08-08T21:00:15.3664578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 40%] 2024-08-08T21:00:15.3666107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0264s] [ 40%] 2024-08-08T21:00:15.3667654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 40%] 2024-08-08T21:00:15.3669172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 40%] 2024-08-08T21:00:15.3670841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 40%] 2024-08-08T21:00:15.3672476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0090s] [ 40%] 2024-08-08T21:00:15.3674040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0090s] [ 40%] 2024-08-08T21:00:15.3675553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 40%] 2024-08-08T21:00:15.3677098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 40%] 2024-08-08T21:00:15.3678634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0073s] [ 40%] 2024-08-08T21:00:15.3680166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 40%] 2024-08-08T21:00:15.3681787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 40%] 2024-08-08T21:00:15.3683406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 40%] 2024-08-08T21:00:15.3684951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0097s] [ 40%] 2024-08-08T21:00:15.3686492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 40%] 2024-08-08T21:00:15.3688065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 40%] 2024-08-08T21:00:15.3689630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 40%] 2024-08-08T21:00:15.3691141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0071s] [ 40%] 2024-08-08T21:00:15.3692689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 40%] 2024-08-08T21:00:15.3694257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 40%] 2024-08-08T21:00:15.3695850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 40%] 2024-08-08T21:00:15.3697394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0091s] [ 40%] 2024-08-08T21:00:15.3698933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0090s] [ 40%] 2024-08-08T21:00:15.3700459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 40%] 2024-08-08T21:00:15.3702007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 40%] 2024-08-08T21:00:15.3703524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 40%] 2024-08-08T21:00:15.3705118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 40%] 2024-08-08T21:00:15.3706708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 40%] 2024-08-08T21:00:15.3708239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 40%] 2024-08-08T21:00:15.3709795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0100s] [ 40%] 2024-08-08T21:00:15.3711346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 40%] 2024-08-08T21:00:15.3712909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 40%] 2024-08-08T21:00:15.3714455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 40%] 2024-08-08T21:00:15.3715998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 40%] 2024-08-08T21:00:15.3717584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 40%] 2024-08-08T21:00:15.3719166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 40%] 2024-08-08T21:00:15.3720688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 40%] 2024-08-08T21:00:15.3722235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0095s] [ 40%] 2024-08-08T21:00:15.3723774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 40%] 2024-08-08T21:00:15.3725383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 40%] 2024-08-08T21:00:15.3726837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 40%] 2024-08-08T21:00:15.3728400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 40%] 2024-08-08T21:00:15.3729998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 40%] 2024-08-08T21:00:15.3731508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 40%] 2024-08-08T21:00:15.3733049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 40%] 2024-08-08T21:00:15.3734585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 40%] 2024-08-08T21:00:15.3736138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 40%] 2024-08-08T21:00:15.3737654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 40%] 2024-08-08T21:00:15.3739186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 40%] 2024-08-08T21:00:15.3740754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 40%] 2024-08-08T21:00:15.3742333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 40%] 2024-08-08T21:00:15.3743867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 41%] 2024-08-08T21:00:15.3745397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 41%] 2024-08-08T21:00:15.3746947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0090s] [ 41%] 2024-08-08T21:00:15.3748473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 41%] 2024-08-08T21:00:15.3750002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 41%] 2024-08-08T21:00:15.3751577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 41%] 2024-08-08T21:00:15.3753219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0104s] [ 41%] 2024-08-08T21:00:15.3754756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0110s] [ 41%] 2024-08-08T21:00:15.3756282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 41%] 2024-08-08T21:00:15.3757839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 41%] 2024-08-08T21:00:15.3759621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0143s] [ 41%] 2024-08-08T21:00:15.3761199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0144s] [ 41%] 2024-08-08T21:00:15.3762738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0078s] [ 41%] 2024-08-08T21:00:15.3764388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 41%] 2024-08-08T21:00:15.3765977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 41%] 2024-08-08T21:00:15.3767529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 41%] 2024-08-08T21:00:15.3769046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 41%] 2024-08-08T21:00:15.3770572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 41%] 2024-08-08T21:00:15.3772111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0098s] [ 41%] 2024-08-08T21:00:15.3773643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 41%] 2024-08-08T21:00:15.3775240Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 41%] 2024-08-08T21:00:15.3776852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 41%] 2024-08-08T21:00:15.3778383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0110s] [ 41%] 2024-08-08T21:00:15.3779935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0119s] [ 41%] 2024-08-08T21:00:15.3781459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 41%] 2024-08-08T21:00:15.3782992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 41%] 2024-08-08T21:00:15.3784540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0147s] [ 41%] 2024-08-08T21:00:15.3786083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0149s] [ 41%] 2024-08-08T21:00:15.3787678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0082s] [ 41%] 2024-08-08T21:00:15.3789291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 41%] 2024-08-08T21:00:15.3790817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 41%] 2024-08-08T21:00:15.3792367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 41%] 2024-08-08T21:00:15.3793891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 41%] 2024-08-08T21:00:15.3795433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 41%] 2024-08-08T21:00:15.3797016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0097s] [ 41%] 2024-08-08T21:00:15.3798622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0098s] [ 41%] 2024-08-08T21:00:15.3800144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 41%] 2024-08-08T21:00:15.3801696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 41%] 2024-08-08T21:00:15.3803218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0120s] [ 41%] 2024-08-08T21:00:15.3804756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0130s] [ 41%] 2024-08-08T21:00:15.3806290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 41%] 2024-08-08T21:00:15.3807818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 41%] 2024-08-08T21:00:15.3809367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0157s] [ 41%] 2024-08-08T21:00:15.3810961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0158s] [ 41%] 2024-08-08T21:00:15.3812568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0091s] [ 41%] 2024-08-08T21:00:15.3814114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 41%] 2024-08-08T21:00:15.3816511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0085s] [ 41%] 2024-08-08T21:00:15.3818357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 41%] 2024-08-08T21:00:15.3820194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 41%] 2024-08-08T21:00:15.3821905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 41%] 2024-08-08T21:00:15.3823665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0103s] [ 41%] 2024-08-08T21:00:15.3825222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 41%] 2024-08-08T21:00:15.3826737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0076s] [ 41%] 2024-08-08T21:00:15.3828302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 41%] 2024-08-08T21:00:15.3829806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0103s] [ 41%] 2024-08-08T21:00:15.3831354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0109s] [ 41%] 2024-08-08T21:00:15.3832856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 41%] 2024-08-08T21:00:15.3834451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 41%] 2024-08-08T21:00:15.3836028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0144s] [ 41%] 2024-08-08T21:00:15.3837566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0145s] [ 41%] 2024-08-08T21:00:15.3839100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0078s] [ 41%] 2024-08-08T21:00:15.3840630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 41%] 2024-08-08T21:00:15.3842169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0078s] [ 41%] 2024-08-08T21:00:15.3843704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 41%] 2024-08-08T21:00:15.3845282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 41%] 2024-08-08T21:00:15.3846894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 41%] 2024-08-08T21:00:15.3848417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0092s] [ 41%] 2024-08-08T21:00:15.3849946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 41%] 2024-08-08T21:00:15.3851461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 41%] 2024-08-08T21:00:15.3853058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 41%] 2024-08-08T21:00:15.3854581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 41%] 2024-08-08T21:00:15.3856134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 41%] 2024-08-08T21:00:15.3857695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 41%] 2024-08-08T21:00:15.3859713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 41%] 2024-08-08T21:00:15.3861268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 41%] 2024-08-08T21:00:15.3862819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 41%] 2024-08-08T21:00:15.3864337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 41%] 2024-08-08T21:00:15.3865891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 42%] 2024-08-08T21:00:15.3867397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 42%] 2024-08-08T21:00:15.3869038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 42%] 2024-08-08T21:00:15.3870656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 42%] 2024-08-08T21:00:15.3872179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 42%] 2024-08-08T21:00:15.3873707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 42%] 2024-08-08T21:00:15.3875243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 42%] 2024-08-08T21:00:15.3876781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 42%] 2024-08-08T21:00:15.3878306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 42%] 2024-08-08T21:00:15.3879832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 42%] 2024-08-08T21:00:15.3881364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 42%] 2024-08-08T21:00:15.3882964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 42%] 2024-08-08T21:00:15.3884571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 42%] 2024-08-08T21:00:15.3886097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 42%] 2024-08-08T21:00:15.3887648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 42%] 2024-08-08T21:00:15.3889167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 42%] 2024-08-08T21:00:15.3890727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 42%] 2024-08-08T21:00:15.3892215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 42%] 2024-08-08T21:00:15.3893801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 42%] 2024-08-08T21:00:15.3895370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 42%] 2024-08-08T21:00:15.3896906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 42%] 2024-08-08T21:00:15.3898422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 42%] 2024-08-08T21:00:15.3899961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 42%] 2024-08-08T21:00:15.3901493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 42%] 2024-08-08T21:00:15.3903015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 42%] 2024-08-08T21:00:15.3904543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 42%] 2024-08-08T21:00:15.3906120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 42%] 2024-08-08T21:00:15.3907702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 42%] 2024-08-08T21:00:15.3909220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 42%] 2024-08-08T21:00:15.3910760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 42%] 2024-08-08T21:00:15.3912298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 42%] 2024-08-08T21:00:15.3913838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 42%] 2024-08-08T21:00:15.3915353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 42%] 2024-08-08T21:00:15.3916907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0068s] [ 42%] 2024-08-08T21:00:15.3918501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 42%] 2024-08-08T21:00:15.3920005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 42%] 2024-08-08T21:00:15.3921538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 42%] 2024-08-08T21:00:15.3923060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0079s] [ 42%] 2024-08-08T21:00:15.3924611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 42%] 2024-08-08T21:00:15.3926121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 42%] 2024-08-08T21:00:15.3927662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 42%] 2024-08-08T21:00:15.3929220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 42%] 2024-08-08T21:00:15.3930790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 42%] 2024-08-08T21:00:15.3932319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 42%] 2024-08-08T21:00:15.3933843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 42%] 2024-08-08T21:00:15.3935393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 42%] 2024-08-08T21:00:15.3936923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 42%] 2024-08-08T21:00:15.3938447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 42%] 2024-08-08T21:00:15.3940023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 42%] 2024-08-08T21:00:15.3941597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 42%] 2024-08-08T21:00:15.3943108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 42%] 2024-08-08T21:00:15.3944629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 42%] 2024-08-08T21:00:15.3946154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 42%] 2024-08-08T21:00:15.3947669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 42%] 2024-08-08T21:00:15.3949207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 42%] 2024-08-08T21:00:15.3950719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 42%] 2024-08-08T21:00:15.3952301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 42%] 2024-08-08T21:00:15.3953924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0176s] [ 42%] 2024-08-08T21:00:15.3955477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0192s] [ 42%] 2024-08-08T21:00:15.3956999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0080s] [ 42%] 2024-08-08T21:00:15.3958937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 42%] 2024-08-08T21:00:15.3960558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0244s] [ 42%] 2024-08-08T21:00:15.3962116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0259s] [ 42%] 2024-08-08T21:00:15.3963756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0114s] [ 42%] 2024-08-08T21:00:15.3965386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0114s] [ 42%] 2024-08-08T21:00:15.3966924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0093s] [ 42%] 2024-08-08T21:00:15.3968462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 42%] 2024-08-08T21:00:15.3970000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 42%] 2024-08-08T21:00:15.3971529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 42%] 2024-08-08T21:00:15.3973070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0114s] [ 42%] 2024-08-08T21:00:15.3974606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0114s] [ 42%] 2024-08-08T21:00:15.3976395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0087s] [ 42%] 2024-08-08T21:00:15.3978001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 42%] 2024-08-08T21:00:15.3979517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0180s] [ 42%] 2024-08-08T21:00:15.3981068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0200s] [ 42%] 2024-08-08T21:00:15.3982596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0080s] [ 42%] 2024-08-08T21:00:15.3984142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 42%] 2024-08-08T21:00:15.3985669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0255s] [ 42%] 2024-08-08T21:00:15.3987275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0262s] [ 43%] 2024-08-08T21:00:15.3988870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0117s] [ 43%] 2024-08-08T21:00:15.3990434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0119s] [ 43%] 2024-08-08T21:00:15.3991954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0094s] [ 43%] 2024-08-08T21:00:15.3993490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0099s] [ 43%] 2024-08-08T21:00:15.3995019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0076s] [ 43%] 2024-08-08T21:00:15.3996542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 43%] 2024-08-08T21:00:15.3998083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0113s] [ 43%] 2024-08-08T21:00:15.3999665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 43%] 2024-08-08T21:00:15.4001248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0089s] [ 43%] 2024-08-08T21:00:15.4002772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 43%] 2024-08-08T21:00:15.4004316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0200s] [ 43%] 2024-08-08T21:00:15.4005854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0218s] [ 43%] 2024-08-08T21:00:15.4007387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0082s] [ 43%] 2024-08-08T21:00:15.4008911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 43%] 2024-08-08T21:00:15.4010488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0270s] [ 43%] 2024-08-08T21:00:15.4012105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0276s] [ 43%] 2024-08-08T21:00:15.4013648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0137s] [ 43%] 2024-08-08T21:00:15.4015215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0130s] [ 43%] 2024-08-08T21:00:15.4016741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0099s] [ 43%] 2024-08-08T21:00:15.4018290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 43%] 2024-08-08T21:00:15.4019798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0078s] [ 43%] 2024-08-08T21:00:15.4021335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 43%] 2024-08-08T21:00:15.4022910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0122s] [ 43%] 2024-08-08T21:00:15.4024509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0123s] [ 43%] 2024-08-08T21:00:15.4026044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 43%] 2024-08-08T21:00:15.4027557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 43%] 2024-08-08T21:00:15.4029098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0170s] [ 43%] 2024-08-08T21:00:15.4030621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0182s] [ 43%] 2024-08-08T21:00:15.4032146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 43%] 2024-08-08T21:00:15.4033724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 43%] 2024-08-08T21:00:15.4035329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0244s] [ 43%] 2024-08-08T21:00:15.4036858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0251s] [ 43%] 2024-08-08T21:00:15.4038397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0111s] [ 43%] 2024-08-08T21:00:15.4039936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0111s] [ 43%] 2024-08-08T21:00:15.4041451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0091s] [ 43%] 2024-08-08T21:00:15.4042992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 43%] 2024-08-08T21:00:15.4044506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 43%] 2024-08-08T21:00:15.4046153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 43%] 2024-08-08T21:00:15.4047724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0111s] [ 43%] 2024-08-08T21:00:15.4049267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0109s] [ 43%] 2024-08-08T21:00:15.4050784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0083s] [ 43%] 2024-08-08T21:00:15.4052333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0083s] [ 43%] 2024-08-08T21:00:15.4053888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 43%] 2024-08-08T21:00:15.4055437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 43%] 2024-08-08T21:00:15.4056997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 43%] 2024-08-08T21:00:15.4058993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 43%] 2024-08-08T21:00:15.4060576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 43%] 2024-08-08T21:00:15.4062114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 43%] 2024-08-08T21:00:15.4063655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 43%] 2024-08-08T21:00:15.4065188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 43%] 2024-08-08T21:00:15.4066712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 43%] 2024-08-08T21:00:15.4068245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 43%] 2024-08-08T21:00:15.4069872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 43%] 2024-08-08T21:00:15.4071474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 43%] 2024-08-08T21:00:15.4073004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0080s] [ 43%] 2024-08-08T21:00:15.4074554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 43%] 2024-08-08T21:00:15.4076076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 43%] 2024-08-08T21:00:15.4077615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 43%] 2024-08-08T21:00:15.4079125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 43%] 2024-08-08T21:00:15.4080725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 43%] 2024-08-08T21:00:15.4082306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 43%] 2024-08-08T21:00:15.4083843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 43%] 2024-08-08T21:00:15.4085377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0079s] [ 43%] 2024-08-08T21:00:15.4086943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 43%] 2024-08-08T21:00:15.4088462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 43%] 2024-08-08T21:00:15.4089989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 43%] 2024-08-08T21:00:15.4091521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 43%] 2024-08-08T21:00:15.4093113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 43%] 2024-08-08T21:00:15.4094699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 43%] 2024-08-08T21:00:15.4096226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 43%] 2024-08-08T21:00:15.4097775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0080s] [ 43%] 2024-08-08T21:00:15.4099314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 43%] 2024-08-08T21:00:15.4100849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 43%] 2024-08-08T21:00:15.4102370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 43%] 2024-08-08T21:00:15.4103950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 43%] 2024-08-08T21:00:15.4105525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 43%] 2024-08-08T21:00:15.4107048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 43%] 2024-08-08T21:00:15.4108578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 44%] 2024-08-08T21:00:15.4110115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 44%] 2024-08-08T21:00:15.4111670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 44%] 2024-08-08T21:00:15.4113197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 44%] 2024-08-08T21:00:15.4114742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 44%] 2024-08-08T21:00:15.4116305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 44%] 2024-08-08T21:00:15.4117902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 44%] 2024-08-08T21:00:15.4119419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 44%] 2024-08-08T21:00:15.4120952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 44%] 2024-08-08T21:00:15.4122504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0083s] [ 44%] 2024-08-08T21:00:15.4124033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 44%] 2024-08-08T21:00:15.4125564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 44%] 2024-08-08T21:00:15.4127137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 44%] 2024-08-08T21:00:15.4128719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 44%] 2024-08-08T21:00:15.4130253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 44%] 2024-08-08T21:00:15.4131795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 44%] 2024-08-08T21:00:15.4133329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 44%] 2024-08-08T21:00:15.4134876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 44%] 2024-08-08T21:00:15.4136404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 44%] 2024-08-08T21:00:15.4137921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 44%] 2024-08-08T21:00:15.4139518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 44%] 2024-08-08T21:00:15.4141067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 44%] 2024-08-08T21:00:15.4142612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 44%] 2024-08-08T21:00:15.4144125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 44%] 2024-08-08T21:00:15.4145668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 44%] 2024-08-08T21:00:15.4147177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 44%] 2024-08-08T21:00:15.4148728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 44%] 2024-08-08T21:00:15.4150306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 44%] 2024-08-08T21:00:15.4151886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 44%] 2024-08-08T21:00:15.4153460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 44%] 2024-08-08T21:00:15.4154998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 44%] 2024-08-08T21:00:15.4156540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 44%] 2024-08-08T21:00:15.4158071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 44%] 2024-08-08T21:00:15.4160067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 44%] 2024-08-08T21:00:15.4161575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 44%] 2024-08-08T21:00:15.4163207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 44%] 2024-08-08T21:00:15.4164806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 44%] 2024-08-08T21:00:15.4166329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 44%] 2024-08-08T21:00:15.4167860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 44%] 2024-08-08T21:00:15.4169377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 44%] 2024-08-08T21:00:15.4170920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 44%] 2024-08-08T21:00:15.4172434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 44%] 2024-08-08T21:00:15.4174041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 44%] 2024-08-08T21:00:15.4175630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 44%] 2024-08-08T21:00:15.4177170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 44%] 2024-08-08T21:00:15.4178685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 44%] 2024-08-08T21:00:15.4180239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 44%] 2024-08-08T21:00:15.4181755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 44%] 2024-08-08T21:00:15.4183276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 44%] 2024-08-08T21:00:15.4184809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 44%] 2024-08-08T21:00:15.4186392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 44%] 2024-08-08T21:00:15.4187974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 44%] 2024-08-08T21:00:15.4189501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 44%] 2024-08-08T21:00:15.4191019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 44%] 2024-08-08T21:00:15.4192553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 44%] 2024-08-08T21:00:15.4194081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 44%] 2024-08-08T21:00:15.4195598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 44%] 2024-08-08T21:00:15.4197173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 44%] 2024-08-08T21:00:15.4198756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 44%] 2024-08-08T21:00:15.4200272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 44%] 2024-08-08T21:00:15.4201818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 44%] 2024-08-08T21:00:15.4203332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 44%] 2024-08-08T21:00:15.4204888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 44%] 2024-08-08T21:00:15.4206399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 44%] 2024-08-08T21:00:15.4207941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 44%] 2024-08-08T21:00:15.4209502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 44%] 2024-08-08T21:00:15.4211098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 44%] 2024-08-08T21:00:15.4212606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 44%] 2024-08-08T21:00:15.4214132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 44%] 2024-08-08T21:00:15.4215660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 44%] 2024-08-08T21:00:15.4217192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 44%] 2024-08-08T21:00:15.4218718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 44%] 2024-08-08T21:00:15.4220298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 44%] 2024-08-08T21:00:15.4221865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0083s] [ 44%] 2024-08-08T21:00:15.4223390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 44%] 2024-08-08T21:00:15.4224919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 44%] 2024-08-08T21:00:15.4226450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 44%] 2024-08-08T21:00:15.4227984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 44%] 2024-08-08T21:00:15.4229512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 45%] 2024-08-08T21:00:15.4231031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 45%] 2024-08-08T21:00:15.4232614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 45%] 2024-08-08T21:00:15.4234197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 45%] 2024-08-08T21:00:15.4235747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 45%] 2024-08-08T21:00:15.4237261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 45%] 2024-08-08T21:00:15.4238814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 45%] 2024-08-08T21:00:15.4240313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 45%] 2024-08-08T21:00:15.4241848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 45%] 2024-08-08T21:00:15.4243389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 45%] 2024-08-08T21:00:15.4244954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 45%] 2024-08-08T21:00:15.4246484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 45%] 2024-08-08T21:00:15.4248008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 45%] 2024-08-08T21:00:15.4249540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 45%] 2024-08-08T21:00:15.4251064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 45%] 2024-08-08T21:00:15.4252607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0103s] [ 45%] 2024-08-08T21:00:15.4254188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 45%] 2024-08-08T21:00:15.4255777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 45%] 2024-08-08T21:00:15.4257355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 45%] 2024-08-08T21:00:15.4259320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0142s] [ 45%] 2024-08-08T21:00:15.4260900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0145s] [ 45%] 2024-08-08T21:00:15.4262450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0083s] [ 45%] 2024-08-08T21:00:15.4264017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 45%] 2024-08-08T21:00:15.4265537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0099s] [ 45%] 2024-08-08T21:00:15.4267169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 45%] 2024-08-08T21:00:15.4268764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 45%] 2024-08-08T21:00:15.4270303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 45%] 2024-08-08T21:00:15.4271824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0137s] [ 45%] 2024-08-08T21:00:15.4273383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0138s] [ 45%] 2024-08-08T21:00:15.4274916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0083s] [ 45%] 2024-08-08T21:00:15.4276466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 45%] 2024-08-08T21:00:15.4277984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0104s] [ 45%] 2024-08-08T21:00:15.4279589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0111s] [ 45%] 2024-08-08T21:00:15.4281188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 45%] 2024-08-08T21:00:15.4282717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 45%] 2024-08-08T21:00:15.4284270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0145s] [ 45%] 2024-08-08T21:00:15.4285825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0146s] [ 45%] 2024-08-08T21:00:15.4287391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 45%] 2024-08-08T21:00:15.4288958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 45%] 2024-08-08T21:00:15.4290530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0101s] [ 45%] 2024-08-08T21:00:15.4292112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 45%] 2024-08-08T21:00:15.4293627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 45%] 2024-08-08T21:00:15.4295169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 45%] 2024-08-08T21:00:15.4296702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0138s] [ 45%] 2024-08-08T21:00:15.4298265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0142s] [ 45%] 2024-08-08T21:00:15.4299780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0086s] [ 45%] 2024-08-08T21:00:15.4301332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 45%] 2024-08-08T21:00:15.4302906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0115s] [ 45%] 2024-08-08T21:00:15.4304504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0121s] [ 45%] 2024-08-08T21:00:15.4306022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 45%] 2024-08-08T21:00:15.4307574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 45%] 2024-08-08T21:00:15.4309112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0153s] [ 45%] 2024-08-08T21:00:15.4310663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0153s] [ 45%] 2024-08-08T21:00:15.4312224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 45%] 2024-08-08T21:00:15.4313823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0094s] [ 45%] 2024-08-08T21:00:15.4315418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0109s] [ 45%] 2024-08-08T21:00:15.4316952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 45%] 2024-08-08T21:00:15.4318487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 45%] 2024-08-08T21:00:15.4320014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 45%] 2024-08-08T21:00:15.4321577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0147s] [ 45%] 2024-08-08T21:00:15.4323104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0146s] [ 45%] 2024-08-08T21:00:15.4324623Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0091s] [ 45%] 2024-08-08T21:00:15.4326234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 45%] 2024-08-08T21:00:15.4327802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0099s] [ 45%] 2024-08-08T21:00:15.4329347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 45%] 2024-08-08T21:00:15.4330865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 45%] 2024-08-08T21:00:15.4332409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 45%] 2024-08-08T21:00:15.4333930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0140s] [ 45%] 2024-08-08T21:00:15.4335482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0147s] [ 45%] 2024-08-08T21:00:15.4337047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 45%] 2024-08-08T21:00:15.4338650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 45%] 2024-08-08T21:00:15.4340158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0097s] [ 45%] 2024-08-08T21:00:15.4341686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0098s] [ 45%] 2024-08-08T21:00:15.4343222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 45%] 2024-08-08T21:00:15.4344757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 45%] 2024-08-08T21:00:15.4346294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0135s] [ 45%] 2024-08-08T21:00:15.4347828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0137s] [ 45%] 2024-08-08T21:00:15.4349416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 45%] 2024-08-08T21:00:15.4350992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 46%] 2024-08-08T21:00:15.4352526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0159s] [ 46%] 2024-08-08T21:00:15.4354103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0169s] [ 46%] 2024-08-08T21:00:15.4355652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 46%] 2024-08-08T21:00:15.4357190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 46%] 2024-08-08T21:00:15.4359099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0230s] [ 46%] 2024-08-08T21:00:15.4360790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0236s] [ 46%] 2024-08-08T21:00:15.4362433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0108s] [ 46%] 2024-08-08T21:00:15.4363980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0108s] [ 46%] 2024-08-08T21:00:15.4365533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0131s] [ 46%] 2024-08-08T21:00:15.4367063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0133s] [ 46%] 2024-08-08T21:00:15.4368577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 46%] 2024-08-08T21:00:15.4370116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 46%] 2024-08-08T21:00:15.4371642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0181s] [ 46%] 2024-08-08T21:00:15.4373246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0183s] [ 46%] 2024-08-08T21:00:15.4374848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 46%] 2024-08-08T21:00:15.4376376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 46%] 2024-08-08T21:00:15.4377909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0162s] [ 46%] 2024-08-08T21:00:15.4379446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0179s] [ 46%] 2024-08-08T21:00:15.4380982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 46%] 2024-08-08T21:00:15.4382508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 46%] 2024-08-08T21:00:15.4384098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0236s] [ 46%] 2024-08-08T21:00:15.4385696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0241s] [ 46%] 2024-08-08T21:00:15.4387260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0111s] [ 46%] 2024-08-08T21:00:15.4388815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0110s] [ 46%] 2024-08-08T21:00:15.4390344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0135s] [ 46%] 2024-08-08T21:00:15.4391897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0141s] [ 46%] 2024-08-08T21:00:15.4393409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 46%] 2024-08-08T21:00:15.4394954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 46%] 2024-08-08T21:00:15.4396533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0185s] [ 46%] 2024-08-08T21:00:15.4398136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0187s] [ 46%] 2024-08-08T21:00:15.4399655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0097s] [ 46%] 2024-08-08T21:00:15.4401204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0096s] [ 46%] 2024-08-08T21:00:15.4402730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0180s] [ 46%] 2024-08-08T21:00:15.4404278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0198s] [ 46%] 2024-08-08T21:00:15.4405792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0080s] [ 46%] 2024-08-08T21:00:15.4407375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 46%] 2024-08-08T21:00:15.4409048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0252s] [ 46%] 2024-08-08T21:00:15.4410588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0255s] [ 46%] 2024-08-08T21:00:15.4412144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0130s] [ 46%] 2024-08-08T21:00:15.4413700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 46%] 2024-08-08T21:00:15.4415239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0143s] [ 46%] 2024-08-08T21:00:15.4416762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0149s] [ 46%] 2024-08-08T21:00:15.4418294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0076s] [ 46%] 2024-08-08T21:00:15.4419860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 46%] 2024-08-08T21:00:15.4421432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0197s] [ 46%] 2024-08-08T21:00:15.4422995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0198s] [ 46%] 2024-08-08T21:00:15.4424515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0109s] [ 46%] 2024-08-08T21:00:15.4426066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0110s] [ 46%] 2024-08-08T21:00:15.4427581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0153s] [ 46%] 2024-08-08T21:00:15.4429135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0159s] [ 46%] 2024-08-08T21:00:15.4430677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 46%] 2024-08-08T21:00:15.4432264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 46%] 2024-08-08T21:00:15.4433777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0228s] [ 46%] 2024-08-08T21:00:15.4435335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0231s] [ 46%] 2024-08-08T21:00:15.4436864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0106s] [ 46%] 2024-08-08T21:00:15.4438395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0106s] [ 46%] 2024-08-08T21:00:15.4439921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0125s] [ 46%] 2024-08-08T21:00:15.4441454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0127s] [ 46%] 2024-08-08T21:00:15.4443027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 46%] 2024-08-08T21:00:15.4444595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 46%] 2024-08-08T21:00:15.4446129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0175s] [ 46%] 2024-08-08T21:00:15.4447666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0178s] [ 46%] 2024-08-08T21:00:15.4449206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 46%] 2024-08-08T21:00:15.4450719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 46%] 2024-08-08T21:00:15.4452225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0071s] [ 46%] 2024-08-08T21:00:15.4453864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 46%] 2024-08-08T21:00:15.4456245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 46%] 2024-08-08T21:00:15.4457787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 46%] 2024-08-08T21:00:15.4459708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0090s] [ 46%] 2024-08-08T21:00:15.4461302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 46%] 2024-08-08T21:00:15.4462823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 46%] 2024-08-08T21:00:15.4464365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 46%] 2024-08-08T21:00:15.4465873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0074s] [ 46%] 2024-08-08T21:00:15.4467514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 46%] 2024-08-08T21:00:15.4469101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 46%] 2024-08-08T21:00:15.4470630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 46%] 2024-08-08T21:00:15.4472173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 46%] 2024-08-08T21:00:15.4473723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 47%] 2024-08-08T21:00:15.4475256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 47%] 2024-08-08T21:00:15.4476777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 47%] 2024-08-08T21:00:15.4478403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0073s] [ 47%] 2024-08-08T21:00:15.4479972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 47%] 2024-08-08T21:00:15.4481508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 47%] 2024-08-08T21:00:15.4483041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 47%] 2024-08-08T21:00:15.4484573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0092s] [ 47%] 2024-08-08T21:00:15.4486126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0093s] [ 47%] 2024-08-08T21:00:15.4487639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 47%] 2024-08-08T21:00:15.4489206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 47%] 2024-08-08T21:00:15.4490738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 47%] 2024-08-08T21:00:15.4492331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 47%] 2024-08-08T21:00:15.4493846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 47%] 2024-08-08T21:00:15.4495382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 47%] 2024-08-08T21:00:15.4496905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0099s] [ 47%] 2024-08-08T21:00:15.4498465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0098s] [ 47%] 2024-08-08T21:00:15.4499980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 47%] 2024-08-08T21:00:15.4501566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 47%] 2024-08-08T21:00:15.4503158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0080s] [ 47%] 2024-08-08T21:00:15.4504685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 47%] 2024-08-08T21:00:15.4506217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 47%] 2024-08-08T21:00:15.4507751Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 47%] 2024-08-08T21:00:15.4509304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0103s] [ 47%] 2024-08-08T21:00:15.4510832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 47%] 2024-08-08T21:00:15.4512367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 47%] 2024-08-08T21:00:15.4513948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 47%] 2024-08-08T21:00:15.4515505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0080s] [ 47%] 2024-08-08T21:00:15.4517051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 47%] 2024-08-08T21:00:15.4518573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 47%] 2024-08-08T21:00:15.4520117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 47%] 2024-08-08T21:00:15.4521655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0104s] [ 47%] 2024-08-08T21:00:15.4523185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 47%] 2024-08-08T21:00:15.4524754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 47%] 2024-08-08T21:00:15.4526352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 47%] 2024-08-08T21:00:15.4527861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0072s] [ 47%] 2024-08-08T21:00:15.4529402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 47%] 2024-08-08T21:00:15.4530921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 47%] 2024-08-08T21:00:15.4532463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 47%] 2024-08-08T21:00:15.4533988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0091s] [ 47%] 2024-08-08T21:00:15.4535529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 47%] 2024-08-08T21:00:15.4537113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 47%] 2024-08-08T21:00:15.4538687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 47%] 2024-08-08T21:00:15.4540206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0073s] [ 47%] 2024-08-08T21:00:15.4541728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 47%] 2024-08-08T21:00:15.4543264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 47%] 2024-08-08T21:00:15.4544783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 47%] 2024-08-08T21:00:15.4546294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0092s] [ 47%] 2024-08-08T21:00:15.4547893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 47%] 2024-08-08T21:00:15.4549468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 47%] 2024-08-08T21:00:15.4551006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 47%] 2024-08-08T21:00:15.4552536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0277s] [ 47%] 2024-08-08T21:00:15.4554142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0305s] [ 47%] 2024-08-08T21:00:15.4555677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0097s] [ 47%] 2024-08-08T21:00:15.4557229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0096s] [ 47%] 2024-08-08T21:00:15.4559164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0421s] [ 47%] 2024-08-08T21:00:15.4560853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0438s] [ 47%] 2024-08-08T21:00:15.4562471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0173s] [ 47%] 2024-08-08T21:00:15.4564040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0165s] [ 47%] 2024-08-08T21:00:15.4565584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0149s] [ 47%] 2024-08-08T21:00:15.4567095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0152s] [ 47%] 2024-08-08T21:00:15.4568626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0089s] [ 47%] 2024-08-08T21:00:15.4570147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0089s] [ 47%] 2024-08-08T21:00:15.4571746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0203s] [ 47%] 2024-08-08T21:00:15.4573348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0206s] [ 47%] 2024-08-08T21:00:15.4574878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0114s] [ 47%] 2024-08-08T21:00:15.4576403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0116s] [ 47%] 2024-08-08T21:00:15.4577940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0287s] [ 47%] 2024-08-08T21:00:15.4579490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0322s] [ 47%] 2024-08-08T21:00:15.4580985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0100s] [ 47%] 2024-08-08T21:00:15.4582531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 47%] 2024-08-08T21:00:15.4584103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0426s] [ 47%] 2024-08-08T21:00:15.4585731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0445s] [ 47%] 2024-08-08T21:00:15.4587275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0171s] [ 47%] 2024-08-08T21:00:15.4588844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0174s] [ 47%] 2024-08-08T21:00:15.4590373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0149s] [ 47%] 2024-08-08T21:00:15.4591920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0159s] [ 47%] 2024-08-08T21:00:15.4593431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0094s] [ 48%] 2024-08-08T21:00:15.4594996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 48%] 2024-08-08T21:00:15.4596607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0208s] [ 48%] 2024-08-08T21:00:15.4598128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0211s] [ 48%] 2024-08-08T21:00:15.4599659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0120s] [ 48%] 2024-08-08T21:00:15.4601188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0118s] [ 48%] 2024-08-08T21:00:15.4602738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0320s] [ 48%] 2024-08-08T21:00:15.4604253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0357s] [ 48%] 2024-08-08T21:00:15.4605786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0107s] [ 48%] 2024-08-08T21:00:15.4607355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0108s] [ 48%] 2024-08-08T21:00:15.4608947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0457s] [ 48%] 2024-08-08T21:00:15.4610483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0468s] [ 48%] 2024-08-08T21:00:15.4612028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0215s] [ 48%] 2024-08-08T21:00:15.4613601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0216s] [ 48%] 2024-08-08T21:00:15.4615122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0168s] [ 48%] 2024-08-08T21:00:15.4616665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0178s] [ 48%] 2024-08-08T21:00:15.4618218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0100s] [ 48%] 2024-08-08T21:00:15.4619812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0098s] [ 48%] 2024-08-08T21:00:15.4621400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0223s] [ 48%] 2024-08-08T21:00:15.4622880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0225s] [ 48%] 2024-08-08T21:00:15.4624403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0137s] [ 48%] 2024-08-08T21:00:15.4625962Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0136s] [ 48%] 2024-08-08T21:00:15.4627476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0269s] [ 48%] 2024-08-08T21:00:15.4629003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0289s] [ 48%] 2024-08-08T21:00:15.4630639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0095s] [ 48%] 2024-08-08T21:00:15.4632148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0094s] [ 48%] 2024-08-08T21:00:15.4633684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0415s] [ 48%] 2024-08-08T21:00:15.4635215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0430s] [ 48%] 2024-08-08T21:00:15.4636754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0163s] [ 48%] 2024-08-08T21:00:15.4638292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0162s] [ 48%] 2024-08-08T21:00:15.4639817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0143s] [ 48%] 2024-08-08T21:00:15.4641441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0146s] [ 48%] 2024-08-08T21:00:15.4642958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0085s] [ 48%] 2024-08-08T21:00:15.4644500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 48%] 2024-08-08T21:00:15.4646018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0198s] [ 48%] 2024-08-08T21:00:15.4647571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0199s] [ 48%] 2024-08-08T21:00:15.4649086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0113s] [ 48%] 2024-08-08T21:00:15.4650621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0113s] [ 48%] 2024-08-08T21:00:15.4652146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0076s] [ 48%] 2024-08-08T21:00:15.4653772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 48%] 2024-08-08T21:00:15.4655338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 48%] 2024-08-08T21:00:15.4656880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 48%] 2024-08-08T21:00:15.4658788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0103s] [ 48%] 2024-08-08T21:00:15.4660378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 48%] 2024-08-08T21:00:15.4661924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 48%] 2024-08-08T21:00:15.4663454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 48%] 2024-08-08T21:00:15.4665071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0080s] [ 48%] 2024-08-08T21:00:15.4666679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 48%] 2024-08-08T21:00:15.4668211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 48%] 2024-08-08T21:00:15.4669735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 48%] 2024-08-08T21:00:15.4671300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0105s] [ 48%] 2024-08-08T21:00:15.4672829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 48%] 2024-08-08T21:00:15.4674343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 48%] 2024-08-08T21:00:15.4675885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 48%] 2024-08-08T21:00:15.4677480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0078s] [ 48%] 2024-08-08T21:00:15.4679089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 48%] 2024-08-08T21:00:15.4680604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 48%] 2024-08-08T21:00:15.4682141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 48%] 2024-08-08T21:00:15.4683672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0102s] [ 48%] 2024-08-08T21:00:15.4685238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 48%] 2024-08-08T21:00:15.4686755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 48%] 2024-08-08T21:00:15.4688357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 48%] 2024-08-08T21:00:15.4689922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0082s] [ 48%] 2024-08-08T21:00:15.4691462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 48%] 2024-08-08T21:00:15.4693007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 48%] 2024-08-08T21:00:15.4694524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 48%] 2024-08-08T21:00:15.4696077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0106s] [ 48%] 2024-08-08T21:00:15.4697606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 48%] 2024-08-08T21:00:15.4699140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0077s] [ 48%] 2024-08-08T21:00:15.4700713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 48%] 2024-08-08T21:00:15.4702290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0082s] [ 48%] 2024-08-08T21:00:15.4703810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 48%] 2024-08-08T21:00:15.4705326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 48%] 2024-08-08T21:00:15.4706873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 48%] 2024-08-08T21:00:15.4708398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0111s] [ 48%] 2024-08-08T21:00:15.4709952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0108s] [ 48%] 2024-08-08T21:00:15.4711513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0079s] [ 48%] 2024-08-08T21:00:15.4713113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 48%] 2024-08-08T21:00:15.4714620Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0085s] [ 49%] 2024-08-08T21:00:15.4716163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 49%] 2024-08-08T21:00:15.4717684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0068s] [ 49%] 2024-08-08T21:00:15.4719235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 49%] 2024-08-08T21:00:15.4720750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0112s] [ 49%] 2024-08-08T21:00:15.4722282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0110s] [ 49%] 2024-08-08T21:00:15.4723864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 49%] 2024-08-08T21:00:15.4725434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 49%] 2024-08-08T21:00:15.4726960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 49%] 2024-08-08T21:00:15.4728497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 49%] 2024-08-08T21:00:15.4730044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 49%] 2024-08-08T21:00:15.4731574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 49%] 2024-08-08T21:00:15.4733111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0100s] [ 49%] 2024-08-08T21:00:15.4734680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 49%] 2024-08-08T21:00:15.4736267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 49%] 2024-08-08T21:00:15.4737787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 49%] 2024-08-08T21:00:15.4739289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0080s] [ 49%] 2024-08-08T21:00:15.4740835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 49%] 2024-08-08T21:00:15.4742357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 49%] 2024-08-08T21:00:15.4743889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 49%] 2024-08-08T21:00:15.4745402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0104s] [ 49%] 2024-08-08T21:00:15.4746996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 49%] 2024-08-08T21:00:15.4748561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 49%] 2024-08-08T21:00:15.4750109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 49%] 2024-08-08T21:00:15.4751626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0072s] [ 49%] 2024-08-08T21:00:15.4753215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 49%] 2024-08-08T21:00:15.4754754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 49%] 2024-08-08T21:00:15.4756280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 49%] 2024-08-08T21:00:15.4757873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0098s] [ 49%] 2024-08-08T21:00:15.4759859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 49%] 2024-08-08T21:00:15.4761411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 49%] 2024-08-08T21:00:15.4762933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 49%] 2024-08-08T21:00:15.4764457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 49%] 2024-08-08T21:00:15.4765995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 49%] 2024-08-08T21:00:15.4767525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 49%] 2024-08-08T21:00:15.4769046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 49%] 2024-08-08T21:00:15.4770660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0099s] [ 49%] 2024-08-08T21:00:15.4772282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 49%] 2024-08-08T21:00:15.4773796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 49%] 2024-08-08T21:00:15.4775338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 49%] 2024-08-08T21:00:15.4776858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 49%] 2024-08-08T21:00:15.4778404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 49%] 2024-08-08T21:00:15.4779915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 49%] 2024-08-08T21:00:15.4781586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 49%] 2024-08-08T21:00:15.4783184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0099s] [ 49%] 2024-08-08T21:00:15.4784710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0100s] [ 49%] 2024-08-08T21:00:15.4786243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 49%] 2024-08-08T21:00:15.4787771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 49%] 2024-08-08T21:00:15.4789298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 49%] 2024-08-08T21:00:15.4790824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 49%] 2024-08-08T21:00:15.4792360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 49%] 2024-08-08T21:00:15.4793921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 49%] 2024-08-08T21:00:15.4795495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0103s] [ 49%] 2024-08-08T21:00:15.4797017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 49%] 2024-08-08T21:00:15.4798557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 49%] 2024-08-08T21:00:15.4800089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 49%] 2024-08-08T21:00:15.4801606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0081s] [ 49%] 2024-08-08T21:00:15.4803155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 49%] 2024-08-08T21:00:15.4804709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 49%] 2024-08-08T21:00:15.4806300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 49%] 2024-08-08T21:00:15.4807822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0105s] [ 49%] 2024-08-08T21:00:15.4809362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 49%] 2024-08-08T21:00:15.4810879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 49%] 2024-08-08T21:00:15.4812425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 49%] 2024-08-08T21:00:15.4813926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0085s] [ 49%] 2024-08-08T21:00:15.4815447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 49%] 2024-08-08T21:00:15.4817021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 49%] 2024-08-08T21:00:15.4818596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 49%] 2024-08-08T21:00:15.4820119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0108s] [ 49%] 2024-08-08T21:00:15.4821651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0108s] [ 49%] 2024-08-08T21:00:15.4823191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 49%] 2024-08-08T21:00:15.4824715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 49%] 2024-08-08T21:00:15.4826243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0070s] [ 49%] 2024-08-08T21:00:15.4827808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 49%] 2024-08-08T21:00:15.4829390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 49%] 2024-08-08T21:00:15.4830915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 49%] 2024-08-08T21:00:15.4832431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0096s] [ 49%] 2024-08-08T21:00:15.4833985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 49%] 2024-08-08T21:00:15.4835515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 50%] 2024-08-08T21:00:15.4837054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 50%] 2024-08-08T21:00:15.4838551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0076s] [ 50%] 2024-08-08T21:00:15.4840128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 50%] 2024-08-08T21:00:15.4841680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 50%] 2024-08-08T21:00:15.4843215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 50%] 2024-08-08T21:00:15.4844728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 50%] 2024-08-08T21:00:15.4846255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 50%] 2024-08-08T21:00:15.4847791Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 50%] 2024-08-08T21:00:15.4849307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 50%] 2024-08-08T21:00:15.4850893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0052s] [ 50%] 2024-08-08T21:00:15.4852473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4854049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0050s] [ 50%] 2024-08-08T21:00:15.4855573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 50%] 2024-08-08T21:00:15.4857117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 50%] 2024-08-08T21:00:15.4859030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 50%] 2024-08-08T21:00:15.4860643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 50%] 2024-08-08T21:00:15.4862167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 50%] 2024-08-08T21:00:15.4863768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 50%] 2024-08-08T21:00:15.4865384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 50%] 2024-08-08T21:00:15.4866894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 50%] 2024-08-08T21:00:15.4868437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4869960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 50%] 2024-08-08T21:00:15.4871508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 50%] 2024-08-08T21:00:15.4873021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 50%] 2024-08-08T21:00:15.4874617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 50%] 2024-08-08T21:00:15.4876199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4877737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4879267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 50%] 2024-08-08T21:00:15.4880795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 50%] 2024-08-08T21:00:15.4882338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 50%] 2024-08-08T21:00:15.4883871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 50%] 2024-08-08T21:00:15.4885403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 50%] 2024-08-08T21:00:15.4886973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 50%] 2024-08-08T21:00:15.4888544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 50%] 2024-08-08T21:00:15.4890064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 50%] 2024-08-08T21:00:15.4891591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 50%] 2024-08-08T21:00:15.4893118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4894641Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 50%] 2024-08-08T21:00:15.4896213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 50%] 2024-08-08T21:00:15.4897748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 50%] 2024-08-08T21:00:15.4899350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 50%] 2024-08-08T21:00:15.4900874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4902419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 50%] 2024-08-08T21:00:15.4903936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 50%] 2024-08-08T21:00:15.4905489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 50%] 2024-08-08T21:00:15.4907013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 50%] 2024-08-08T21:00:15.4908541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 50%] 2024-08-08T21:00:15.4910124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 50%] 2024-08-08T21:00:15.4911706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 50%] 2024-08-08T21:00:15.4913223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 50%] 2024-08-08T21:00:15.4914745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 50%] 2024-08-08T21:00:15.4916274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4917802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 50%] 2024-08-08T21:00:15.4919333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 50%] 2024-08-08T21:00:15.4920902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 50%] 2024-08-08T21:00:15.4922491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 50%] 2024-08-08T21:00:15.4924024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 50%] 2024-08-08T21:00:15.4925538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4927087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 50%] 2024-08-08T21:00:15.4928609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0050s] [ 50%] 2024-08-08T21:00:15.4930156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 50%] 2024-08-08T21:00:15.4931669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 50%] 2024-08-08T21:00:15.4933272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 50%] 2024-08-08T21:00:15.4934857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 50%] 2024-08-08T21:00:15.4936402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 50%] 2024-08-08T21:00:15.4937899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 50%] 2024-08-08T21:00:15.4939423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 50%] 2024-08-08T21:00:15.4940956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4942474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 50%] 2024-08-08T21:00:15.4944049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 50%] 2024-08-08T21:00:15.4945629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 50%] 2024-08-08T21:00:15.4947163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 50%] 2024-08-08T21:00:15.4948681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 50%] 2024-08-08T21:00:15.4950217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 50%] 2024-08-08T21:00:15.4951752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 50%] 2024-08-08T21:00:15.4953330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 50%] 2024-08-08T21:00:15.4954861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 50%] 2024-08-08T21:00:15.4956389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 51%] 2024-08-08T21:00:15.4957991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 51%] 2024-08-08T21:00:15.4959927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 51%] 2024-08-08T21:00:15.4961495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 51%] 2024-08-08T21:00:15.4963010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 51%] 2024-08-08T21:00:15.4964562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 51%] 2024-08-08T21:00:15.4966089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 51%] 2024-08-08T21:00:15.4967605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 51%] 2024-08-08T21:00:15.4969200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 51%] 2024-08-08T21:00:15.4970814Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 51%] 2024-08-08T21:00:15.4972350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 51%] 2024-08-08T21:00:15.4973873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 51%] 2024-08-08T21:00:15.4975412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 51%] 2024-08-08T21:00:15.4976944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 51%] 2024-08-08T21:00:15.4978479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 51%] 2024-08-08T21:00:15.4980008Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 51%] 2024-08-08T21:00:15.4981610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 51%] 2024-08-08T21:00:15.4983204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 51%] 2024-08-08T21:00:15.4984740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 51%] 2024-08-08T21:00:15.4986267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 51%] 2024-08-08T21:00:15.4987782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 51%] 2024-08-08T21:00:15.4999308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 51%] 2024-08-08T21:00:15.5001036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 51%] 2024-08-08T21:00:15.5002797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 51%] 2024-08-08T21:00:15.5004477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 51%] 2024-08-08T21:00:15.5006146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 51%] 2024-08-08T21:00:15.5007754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 51%] 2024-08-08T21:00:15.5009426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 51%] 2024-08-08T21:00:15.5011064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0070s] [ 51%] 2024-08-08T21:00:15.5012681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 51%] 2024-08-08T21:00:15.5014336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0050s] [ 51%] 2024-08-08T21:00:15.5016041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 51%] 2024-08-08T21:00:15.5017731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 51%] 2024-08-08T21:00:15.5019354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 51%] 2024-08-08T21:00:15.5021016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 51%] 2024-08-08T21:00:15.5022696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 51%] 2024-08-08T21:00:15.5024320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 51%] 2024-08-08T21:00:15.5025937Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 51%] 2024-08-08T21:00:15.5027589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 51%] 2024-08-08T21:00:15.5029276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 51%] 2024-08-08T21:00:15.5030880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 51%] 2024-08-08T21:00:15.5032544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 51%] 2024-08-08T21:00:15.5034165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 51%] 2024-08-08T21:00:15.5035803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 51%] 2024-08-08T21:00:15.5037401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 51%] 2024-08-08T21:00:15.5039055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 51%] 2024-08-08T21:00:15.5040722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 51%] 2024-08-08T21:00:15.5042436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 51%] 2024-08-08T21:00:15.5044059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 51%] 2024-08-08T21:00:15.5045680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 51%] 2024-08-08T21:00:15.5047337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 51%] 2024-08-08T21:00:15.5048948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 51%] 2024-08-08T21:00:15.5050585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 51%] 2024-08-08T21:00:15.5052257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 51%] 2024-08-08T21:00:15.5053985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 51%] 2024-08-08T21:00:15.5055598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 51%] 2024-08-08T21:00:15.5057237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 51%] 2024-08-08T21:00:15.5059278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 51%] 2024-08-08T21:00:15.5060868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 51%] 2024-08-08T21:00:15.5062431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 51%] 2024-08-08T21:00:15.5063948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 51%] 2024-08-08T21:00:15.5065627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0051s] [ 51%] 2024-08-08T21:00:15.5067203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0050s] [ 51%] 2024-08-08T21:00:15.5068739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 51%] 2024-08-08T21:00:15.5070261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 51%] 2024-08-08T21:00:15.5071819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 51%] 2024-08-08T21:00:15.5073338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 51%] 2024-08-08T21:00:15.5074890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 51%] 2024-08-08T21:00:15.5076460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 51%] 2024-08-08T21:00:15.5078053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 51%] 2024-08-08T21:00:15.5079573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 51%] 2024-08-08T21:00:15.5081088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 51%] 2024-08-08T21:00:15.5082618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 51%] 2024-08-08T21:00:15.5084144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 51%] 2024-08-08T21:00:15.5085669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 51%] 2024-08-08T21:00:15.5087193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 51%] 2024-08-08T21:00:15.5088792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 51%] 2024-08-08T21:00:15.5090359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 51%] 2024-08-08T21:00:15.5091861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 52%] 2024-08-08T21:00:15.5093399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 52%] 2024-08-08T21:00:15.5094931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 52%] 2024-08-08T21:00:15.5096480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 52%] 2024-08-08T21:00:15.5097989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 52%] 2024-08-08T21:00:15.5099582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 52%] 2024-08-08T21:00:15.5101131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5102663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5104161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 52%] 2024-08-08T21:00:15.5105692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 52%] 2024-08-08T21:00:15.5107201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 52%] 2024-08-08T21:00:15.5108719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 52%] 2024-08-08T21:00:15.5110241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 52%] 2024-08-08T21:00:15.5111813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 52%] 2024-08-08T21:00:15.5113403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5114922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5116449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 52%] 2024-08-08T21:00:15.5117980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 52%] 2024-08-08T21:00:15.5119517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 52%] 2024-08-08T21:00:15.5121042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 52%] 2024-08-08T21:00:15.5122594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 52%] 2024-08-08T21:00:15.5124213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 52%] 2024-08-08T21:00:15.5125719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5127250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 52%] 2024-08-08T21:00:15.5128750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 52%] 2024-08-08T21:00:15.5130287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 52%] 2024-08-08T21:00:15.5131797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 52%] 2024-08-08T21:00:15.5133325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 52%] 2024-08-08T21:00:15.5134881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 52%] 2024-08-08T21:00:15.5136451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 52%] 2024-08-08T21:00:15.5137969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5139481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 52%] 2024-08-08T21:00:15.5141007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0050s] [ 52%] 2024-08-08T21:00:15.5142531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 52%] 2024-08-08T21:00:15.5144056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 52%] 2024-08-08T21:00:15.5145621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 52%] 2024-08-08T21:00:15.5147198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 52%] 2024-08-08T21:00:15.5148725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 52%] 2024-08-08T21:00:15.5150258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 52%] 2024-08-08T21:00:15.5151766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 52%] 2024-08-08T21:00:15.5153319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 52%] 2024-08-08T21:00:15.5154850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 52%] 2024-08-08T21:00:15.5156355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 52%] 2024-08-08T21:00:15.5157890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 52%] 2024-08-08T21:00:15.5159923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 52%] 2024-08-08T21:00:15.5161602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 52%] 2024-08-08T21:00:15.5163125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0082s] [ 52%] 2024-08-08T21:00:15.5164683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 52%] 2024-08-08T21:00:15.5166207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 52%] 2024-08-08T21:00:15.5167736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 52%] 2024-08-08T21:00:15.5169273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0091s] [ 52%] 2024-08-08T21:00:15.5170906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 52%] 2024-08-08T21:00:15.5172526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 52%] 2024-08-08T21:00:15.5174050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 52%] 2024-08-08T21:00:15.5175580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5177116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5178643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5180165Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 52%] 2024-08-08T21:00:15.5181700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 52%] 2024-08-08T21:00:15.5183293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 52%] 2024-08-08T21:00:15.5184843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 52%] 2024-08-08T21:00:15.5186384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 52%] 2024-08-08T21:00:15.5187907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0087s] [ 52%] 2024-08-08T21:00:15.5189472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0088s] [ 52%] 2024-08-08T21:00:15.5191016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 52%] 2024-08-08T21:00:15.5192554Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 52%] 2024-08-08T21:00:15.5194102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0099s] [ 52%] 2024-08-08T21:00:15.5195701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0099s] [ 52%] 2024-08-08T21:00:15.5197216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 52%] 2024-08-08T21:00:15.5198745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 52%] 2024-08-08T21:00:15.5200287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 52%] 2024-08-08T21:00:15.5201810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 52%] 2024-08-08T21:00:15.5203335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 52%] 2024-08-08T21:00:15.5204856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 52%] 2024-08-08T21:00:15.5206440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 52%] 2024-08-08T21:00:15.5208011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 52%] 2024-08-08T21:00:15.5209540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 52%] 2024-08-08T21:00:15.5211069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 53%] 2024-08-08T21:00:15.5212613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0104s] [ 53%] 2024-08-08T21:00:15.5214145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 53%] 2024-08-08T21:00:15.5215658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 53%] 2024-08-08T21:00:15.5217261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 53%] 2024-08-08T21:00:15.5218835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0117s] [ 53%] 2024-08-08T21:00:15.5220388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0115s] [ 53%] 2024-08-08T21:00:15.5221892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 53%] 2024-08-08T21:00:15.5223443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 53%] 2024-08-08T21:00:15.5224954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 53%] 2024-08-08T21:00:15.5226495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 53%] 2024-08-08T21:00:15.5228004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 53%] 2024-08-08T21:00:15.5229579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 53%] 2024-08-08T21:00:15.5231154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 53%] 2024-08-08T21:00:15.5232683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 53%] 2024-08-08T21:00:15.5234214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 53%] 2024-08-08T21:00:15.5235753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 53%] 2024-08-08T21:00:15.5237287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 53%] 2024-08-08T21:00:15.5238827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 53%] 2024-08-08T21:00:15.5240380Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 53%] 2024-08-08T21:00:15.5241953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 53%] 2024-08-08T21:00:15.5243488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0088s] [ 53%] 2024-08-08T21:00:15.5245021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 53%] 2024-08-08T21:00:15.5246540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 53%] 2024-08-08T21:00:15.5248098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 53%] 2024-08-08T21:00:15.5249600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 53%] 2024-08-08T21:00:15.5251136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 53%] 2024-08-08T21:00:15.5252685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 53%] 2024-08-08T21:00:15.5254307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 53%] 2024-08-08T21:00:15.5255817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 53%] 2024-08-08T21:00:15.5257353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 53%] 2024-08-08T21:00:15.5259275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 53%] 2024-08-08T21:00:15.5260841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 53%] 2024-08-08T21:00:15.5262375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0052s] [ 53%] 2024-08-08T21:00:15.5263989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 53%] 2024-08-08T21:00:15.5265596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 53%] 2024-08-08T21:00:15.5267126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 53%] 2024-08-08T21:00:15.5268660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 53%] 2024-08-08T21:00:15.5270196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 53%] 2024-08-08T21:00:15.5271771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 53%] 2024-08-08T21:00:15.5273280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 53%] 2024-08-08T21:00:15.5274803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 53%] 2024-08-08T21:00:15.5276394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 53%] 2024-08-08T21:00:15.5277967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 53%] 2024-08-08T21:00:15.5279508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 53%] 2024-08-08T21:00:15.5281031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 53%] 2024-08-08T21:00:15.5282584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 53%] 2024-08-08T21:00:15.5284106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 53%] 2024-08-08T21:00:15.5285651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 53%] 2024-08-08T21:00:15.5287206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 53%] 2024-08-08T21:00:15.5288797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 53%] 2024-08-08T21:00:15.5290308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 53%] 2024-08-08T21:00:15.5291822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 53%] 2024-08-08T21:00:15.5293359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 53%] 2024-08-08T21:00:15.5294903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 53%] 2024-08-08T21:00:15.5296443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 53%] 2024-08-08T21:00:15.5297974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 53%] 2024-08-08T21:00:15.5299546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 53%] 2024-08-08T21:00:15.5301130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 53%] 2024-08-08T21:00:15.5302657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 53%] 2024-08-08T21:00:15.5304175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 53%] 2024-08-08T21:00:15.5305718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 53%] 2024-08-08T21:00:15.5307246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 53%] 2024-08-08T21:00:15.5308759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 53%] 2024-08-08T21:00:15.5310365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 53%] 2024-08-08T21:00:15.5311910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 53%] 2024-08-08T21:00:15.5313459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 53%] 2024-08-08T21:00:15.5314971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 53%] 2024-08-08T21:00:15.5316514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 53%] 2024-08-08T21:00:15.5318041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 53%] 2024-08-08T21:00:15.5319589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 53%] 2024-08-08T21:00:15.5321151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 53%] 2024-08-08T21:00:15.5322685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 53%] 2024-08-08T21:00:15.5324257Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 53%] 2024-08-08T21:00:15.5325771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 53%] 2024-08-08T21:00:15.5327295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 53%] 2024-08-08T21:00:15.5328824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 53%] 2024-08-08T21:00:15.5330358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 53%] 2024-08-08T21:00:15.5331886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 54%] 2024-08-08T21:00:15.5333464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 54%] 2024-08-08T21:00:15.5335058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 54%] 2024-08-08T21:00:15.5336564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5338087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 54%] 2024-08-08T21:00:15.5339595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0050s] [ 54%] 2024-08-08T21:00:15.5341141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 54%] 2024-08-08T21:00:15.5342659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 54%] 2024-08-08T21:00:15.5344202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5345766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5347362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 54%] 2024-08-08T21:00:15.5348857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 54%] 2024-08-08T21:00:15.5350386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 54%] 2024-08-08T21:00:15.5351917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5353462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 54%] 2024-08-08T21:00:15.5354994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 54%] 2024-08-08T21:00:15.5356560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 54%] 2024-08-08T21:00:15.5358145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 54%] 2024-08-08T21:00:15.5360071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 54%] 2024-08-08T21:00:15.5361611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5363136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 54%] 2024-08-08T21:00:15.5364668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 54%] 2024-08-08T21:00:15.5366190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 54%] 2024-08-08T21:00:15.5367723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 54%] 2024-08-08T21:00:15.5369250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5370858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5372469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 54%] 2024-08-08T21:00:15.5373961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 54%] 2024-08-08T21:00:15.5375497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 54%] 2024-08-08T21:00:15.5377000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5378538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 54%] 2024-08-08T21:00:15.5380031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 54%] 2024-08-08T21:00:15.5381625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 54%] 2024-08-08T21:00:15.5383199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 54%] 2024-08-08T21:00:15.5384723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 54%] 2024-08-08T21:00:15.5386250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 54%] 2024-08-08T21:00:15.5387780Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 54%] 2024-08-08T21:00:15.5389305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0050s] [ 54%] 2024-08-08T21:00:15.5390819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 54%] 2024-08-08T21:00:15.5392350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5393935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5395512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5397027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5398553Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 54%] 2024-08-08T21:00:15.5400072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 54%] 2024-08-08T21:00:15.5401572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5403094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 54%] 2024-08-08T21:00:15.5404646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 54%] 2024-08-08T21:00:15.5406236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 54%] 2024-08-08T21:00:15.5407733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 54%] 2024-08-08T21:00:15.5409263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 54%] 2024-08-08T21:00:15.5410773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5412313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5413818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 54%] 2024-08-08T21:00:15.5415335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 54%] 2024-08-08T21:00:15.5416921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 54%] 2024-08-08T21:00:15.5418501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 54%] 2024-08-08T21:00:15.5420025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 54%] 2024-08-08T21:00:15.5421547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 54%] 2024-08-08T21:00:15.5423068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 54%] 2024-08-08T21:00:15.5424582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 54%] 2024-08-08T21:00:15.5426121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 54%] 2024-08-08T21:00:15.5427651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 54%] 2024-08-08T21:00:15.5429210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 54%] 2024-08-08T21:00:15.5430744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 54%] 2024-08-08T21:00:15.5432244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 54%] 2024-08-08T21:00:15.5433779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 54%] 2024-08-08T21:00:15.5435284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5436838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5438308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 54%] 2024-08-08T21:00:15.5439885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 54%] 2024-08-08T21:00:15.5441439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5442970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5444481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 54%] 2024-08-08T21:00:15.5446006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 54%] 2024-08-08T21:00:15.5447521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 54%] 2024-08-08T21:00:15.5449016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 54%] 2024-08-08T21:00:15.5450569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 54%] 2024-08-08T21:00:15.5452121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 55%] 2024-08-08T21:00:15.5453689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 55%] 2024-08-08T21:00:15.5455198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 55%] 2024-08-08T21:00:15.5456724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 55%] 2024-08-08T21:00:15.5458266Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 55%] 2024-08-08T21:00:15.5460204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0156s] [ 55%] 2024-08-08T21:00:15.5461776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0165s] [ 55%] 2024-08-08T21:00:15.5463388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0079s] [ 55%] 2024-08-08T21:00:15.5465011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 55%] 2024-08-08T21:00:15.5466543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0232s] [ 55%] 2024-08-08T21:00:15.5468110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0235s] [ 55%] 2024-08-08T21:00:15.5469667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0114s] [ 55%] 2024-08-08T21:00:15.5471234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0114s] [ 55%] 2024-08-08T21:00:15.5472761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0154s] [ 55%] 2024-08-08T21:00:15.5474381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0161s] [ 55%] 2024-08-08T21:00:15.5475980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 55%] 2024-08-08T21:00:15.5477510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 55%] 2024-08-08T21:00:15.5479052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0226s] [ 55%] 2024-08-08T21:00:15.5480592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0229s] [ 55%] 2024-08-08T21:00:15.5482142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0114s] [ 55%] 2024-08-08T21:00:15.5483675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0114s] [ 55%] 2024-08-08T21:00:15.5485211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0163s] [ 55%] 2024-08-08T21:00:15.5486857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0175s] [ 55%] 2024-08-08T21:00:15.5488382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 55%] 2024-08-08T21:00:15.5489911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 55%] 2024-08-08T21:00:15.5491444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0237s] [ 55%] 2024-08-08T21:00:15.5493012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0238s] [ 55%] 2024-08-08T21:00:15.5494566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0118s] [ 55%] 2024-08-08T21:00:15.5496134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0118s] [ 55%] 2024-08-08T21:00:15.5497699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0160s] [ 55%] 2024-08-08T21:00:15.5499302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0172s] [ 55%] 2024-08-08T21:00:15.5500822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0082s] [ 55%] 2024-08-08T21:00:15.5502363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 55%] 2024-08-08T21:00:15.5503913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0230s] [ 55%] 2024-08-08T21:00:15.5505456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0233s] [ 55%] 2024-08-08T21:00:15.5506982Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0116s] [ 55%] 2024-08-08T21:00:15.5508511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0117s] [ 55%] 2024-08-08T21:00:15.5510100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0178s] [ 55%] 2024-08-08T21:00:15.5511687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0197s] [ 55%] 2024-08-08T21:00:15.5513227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0085s] [ 55%] 2024-08-08T21:00:15.5514754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 55%] 2024-08-08T21:00:15.5516309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0253s] [ 55%] 2024-08-08T21:00:15.5517851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0256s] [ 55%] 2024-08-08T21:00:15.5519416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0138s] [ 55%] 2024-08-08T21:00:15.5521011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0140s] [ 55%] 2024-08-08T21:00:15.5522611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0179s] [ 55%] 2024-08-08T21:00:15.5524144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0191s] [ 55%] 2024-08-08T21:00:15.5525664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0087s] [ 55%] 2024-08-08T21:00:15.5527213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 55%] 2024-08-08T21:00:15.5528748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0246s] [ 55%] 2024-08-08T21:00:15.5530303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0249s] [ 55%] 2024-08-08T21:00:15.5531816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0137s] [ 55%] 2024-08-08T21:00:15.5533458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0138s] [ 55%] 2024-08-08T21:00:15.5535028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0156s] [ 55%] 2024-08-08T21:00:15.5536574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0160s] [ 55%] 2024-08-08T21:00:15.5538091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 55%] 2024-08-08T21:00:15.5539624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 55%] 2024-08-08T21:00:15.5541189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0227s] [ 55%] 2024-08-08T21:00:15.5542699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0232s] [ 55%] 2024-08-08T21:00:15.5544305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0113s] [ 55%] 2024-08-08T21:00:15.5545884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0114s] [ 55%] 2024-08-08T21:00:15.5547411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0149s] [ 55%] 2024-08-08T21:00:15.5548942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0155s] [ 55%] 2024-08-08T21:00:15.5550480Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0079s] [ 55%] 2024-08-08T21:00:15.5552012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 55%] 2024-08-08T21:00:15.5553596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0223s] [ 55%] 2024-08-08T21:00:15.5555130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0225s] [ 55%] 2024-08-08T21:00:15.5556703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0112s] [ 55%] 2024-08-08T21:00:15.5558300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 55%] 2024-08-08T21:00:15.5560231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0259s] [ 55%] 2024-08-08T21:00:15.5561794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0288s] [ 55%] 2024-08-08T21:00:15.5563325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0096s] [ 55%] 2024-08-08T21:00:15.5564878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0096s] [ 55%] 2024-08-08T21:00:15.5566404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0405s] [ 55%] 2024-08-08T21:00:15.5568057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0418s] [ 55%] 2024-08-08T21:00:15.5569676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0167s] [ 55%] 2024-08-08T21:00:15.5571222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0166s] [ 55%] 2024-08-08T21:00:15.5572765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0235s] [ 55%] 2024-08-08T21:00:15.5574295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0250s] [ 56%] 2024-08-08T21:00:15.5575833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0096s] [ 56%] 2024-08-08T21:00:15.5577355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0098s] [ 56%] 2024-08-08T21:00:15.5578901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0356s] [ 56%] 2024-08-08T21:00:15.5580494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0364s] [ 56%] 2024-08-08T21:00:15.5582103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0152s] [ 56%] 2024-08-08T21:00:15.5583632Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0151s] [ 56%] 2024-08-08T21:00:15.5585169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0270s] [ 56%] 2024-08-08T21:00:15.5586707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0302s] [ 56%] 2024-08-08T21:00:15.5588226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0100s] [ 56%] 2024-08-08T21:00:15.5589770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0103s] [ 56%] 2024-08-08T21:00:15.5591341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0414s] [ 56%] 2024-08-08T21:00:15.5592947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0422s] [ 56%] 2024-08-08T21:00:15.5594493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0173s] [ 56%] 2024-08-08T21:00:15.5596056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0171s] [ 56%] 2024-08-08T21:00:15.5597584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0242s] [ 56%] 2024-08-08T21:00:15.5599140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0268s] [ 56%] 2024-08-08T21:00:15.5600653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0104s] [ 56%] 2024-08-08T21:00:15.5602196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 56%] 2024-08-08T21:00:15.5603763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0363s] [ 56%] 2024-08-08T21:00:15.5605345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0369s] [ 56%] 2024-08-08T21:00:15.5606881Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0160s] [ 56%] 2024-08-08T21:00:15.5608414Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0157s] [ 56%] 2024-08-08T21:00:15.5609957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0303s] [ 56%] 2024-08-08T21:00:15.5611493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0341s] [ 56%] 2024-08-08T21:00:15.5613035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0111s] [ 56%] 2024-08-08T21:00:15.5614599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0110s] [ 56%] 2024-08-08T21:00:15.5616198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0447s] [ 56%] 2024-08-08T21:00:15.5617733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0452s] [ 56%] 2024-08-08T21:00:15.5619274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0217s] [ 56%] 2024-08-08T21:00:15.5620867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0217s] [ 56%] 2024-08-08T21:00:15.5622377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0273s] [ 56%] 2024-08-08T21:00:15.5623930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0300s] [ 56%] 2024-08-08T21:00:15.5625440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0108s] [ 56%] 2024-08-08T21:00:15.5627028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0107s] [ 56%] 2024-08-08T21:00:15.5628599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0387s] [ 56%] 2024-08-08T21:00:15.5630145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0392s] [ 56%] 2024-08-08T21:00:15.5631664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0193s] [ 56%] 2024-08-08T21:00:15.5633216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0190s] [ 56%] 2024-08-08T21:00:15.5634735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0248s] [ 56%] 2024-08-08T21:00:15.5636260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0266s] [ 56%] 2024-08-08T21:00:15.5637837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0093s] [ 56%] 2024-08-08T21:00:15.5639418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 56%] 2024-08-08T21:00:15.5640953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0399s] [ 56%] 2024-08-08T21:00:15.5642483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0409s] [ 56%] 2024-08-08T21:00:15.5644023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0164s] [ 56%] 2024-08-08T21:00:15.5645560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0162s] [ 56%] 2024-08-08T21:00:15.5647083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0224s] [ 56%] 2024-08-08T21:00:15.5648617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0234s] [ 56%] 2024-08-08T21:00:15.5650196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0095s] [ 56%] 2024-08-08T21:00:15.5652495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0094s] [ 56%] 2024-08-08T21:00:15.5654060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0346s] [ 56%] 2024-08-08T21:00:15.5655611Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0355s] [ 56%] 2024-08-08T21:00:15.5657133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0148s] [ 56%] 2024-08-08T21:00:15.5659047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0151s] [ 56%] 2024-08-08T21:00:15.5660602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0104s] [ 56%] 2024-08-08T21:00:15.5662249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0100s] [ 56%] 2024-08-08T21:00:15.5663846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 56%] 2024-08-08T21:00:15.5665490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 56%] 2024-08-08T21:00:15.5666915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0139s] [ 56%] 2024-08-08T21:00:15.5668456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0137s] [ 56%] 2024-08-08T21:00:15.5670002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0080s] [ 56%] 2024-08-08T21:00:15.5671530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 56%] 2024-08-08T21:00:15.5673059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0102s] [ 56%] 2024-08-08T21:00:15.5674657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 56%] 2024-08-08T21:00:15.5676255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 56%] 2024-08-08T21:00:15.5677774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 56%] 2024-08-08T21:00:15.5679309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0144s] [ 56%] 2024-08-08T21:00:15.5680851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0143s] [ 56%] 2024-08-08T21:00:15.5682398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0083s] [ 56%] 2024-08-08T21:00:15.5683927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 56%] 2024-08-08T21:00:15.5685491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0105s] [ 56%] 2024-08-08T21:00:15.5687085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0104s] [ 56%] 2024-08-08T21:00:15.5688604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 56%] 2024-08-08T21:00:15.5690152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 56%] 2024-08-08T21:00:15.5691678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0142s] [ 56%] 2024-08-08T21:00:15.5693234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0145s] [ 56%] 2024-08-08T21:00:15.5694748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0079s] [ 56%] 2024-08-08T21:00:15.5696290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 57%] 2024-08-08T21:00:15.5697848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0108s] [ 57%] 2024-08-08T21:00:15.5699420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0107s] [ 57%] 2024-08-08T21:00:15.5700945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 57%] 2024-08-08T21:00:15.5702471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 57%] 2024-08-08T21:00:15.5704012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0148s] [ 57%] 2024-08-08T21:00:15.5705545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0145s] [ 57%] 2024-08-08T21:00:15.5707080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0083s] [ 57%] 2024-08-08T21:00:15.5708650Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 57%] 2024-08-08T21:00:15.5710234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0117s] [ 57%] 2024-08-08T21:00:15.5711773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0115s] [ 57%] 2024-08-08T21:00:15.5713307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 57%] 2024-08-08T21:00:15.5714842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 57%] 2024-08-08T21:00:15.5716371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0153s] [ 57%] 2024-08-08T21:00:15.5717923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0155s] [ 57%] 2024-08-08T21:00:15.5719436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 57%] 2024-08-08T21:00:15.5721023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0089s] [ 57%] 2024-08-08T21:00:15.5722586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0119s] [ 57%] 2024-08-08T21:00:15.5724126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0120s] [ 57%] 2024-08-08T21:00:15.5725643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 57%] 2024-08-08T21:00:15.5727193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 57%] 2024-08-08T21:00:15.5728708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0159s] [ 57%] 2024-08-08T21:00:15.5730235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0155s] [ 57%] 2024-08-08T21:00:15.5731824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0089s] [ 57%] 2024-08-08T21:00:15.5733408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 57%] 2024-08-08T21:00:15.5734941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0097s] [ 57%] 2024-08-08T21:00:15.5736463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 57%] 2024-08-08T21:00:15.5737996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0068s] [ 57%] 2024-08-08T21:00:15.5739525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 57%] 2024-08-08T21:00:15.5741056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0136s] [ 57%] 2024-08-08T21:00:15.5742581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0135s] [ 57%] 2024-08-08T21:00:15.5744187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0078s] [ 57%] 2024-08-08T21:00:15.5745773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 57%] 2024-08-08T21:00:15.5747278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0097s] [ 57%] 2024-08-08T21:00:15.5748816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 57%] 2024-08-08T21:00:15.5750332Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0071s] [ 57%] 2024-08-08T21:00:15.5751869Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 57%] 2024-08-08T21:00:15.5753425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0139s] [ 57%] 2024-08-08T21:00:15.5755019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0139s] [ 57%] 2024-08-08T21:00:15.5756584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 57%] 2024-08-08T21:00:15.5758131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 57%] 2024-08-08T21:00:15.5760070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0481s] [ 57%] 2024-08-08T21:00:15.5761636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0541s] [ 57%] 2024-08-08T21:00:15.5763185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0134s] [ 57%] 2024-08-08T21:00:15.5764715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 57%] 2024-08-08T21:00:15.5766259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.2484s] [ 57%] 2024-08-08T21:00:15.5767886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0798s] [ 57%] 2024-08-08T21:00:15.5769524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0274s] [ 57%] 2024-08-08T21:00:15.5771073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0275s] [ 57%] 2024-08-08T21:00:15.5772614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0329s] [ 57%] 2024-08-08T21:00:15.5774155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0356s] [ 57%] 2024-08-08T21:00:15.5775692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0131s] [ 57%] 2024-08-08T21:00:15.5777216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0128s] [ 57%] 2024-08-08T21:00:15.5778793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0499s] [ 57%] 2024-08-08T21:00:15.5780416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0514s] [ 57%] 2024-08-08T21:00:15.5781946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0210s] [ 57%] 2024-08-08T21:00:15.5783493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0206s] [ 57%] 2024-08-08T21:00:15.5785013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0501s] [ 57%] 2024-08-08T21:00:15.5786566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0571s] [ 57%] 2024-08-08T21:00:15.5788081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0146s] [ 57%] 2024-08-08T21:00:15.5789619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0145s] [ 57%] 2024-08-08T21:00:15.5791186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0772s] [ 57%] 2024-08-08T21:00:15.5792797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0805s] [ 57%] 2024-08-08T21:00:15.5794342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0289s] [ 57%] 2024-08-08T21:00:15.5795888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0283s] [ 57%] 2024-08-08T21:00:15.5797437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0344s] [ 57%] 2024-08-08T21:00:15.5798975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0378s] [ 57%] 2024-08-08T21:00:15.5800507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0138s] [ 57%] 2024-08-08T21:00:15.5802073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0136s] [ 57%] 2024-08-08T21:00:15.5803666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0516s] [ 57%] 2024-08-08T21:00:15.5805206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0525s] [ 57%] 2024-08-08T21:00:15.5806742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0219s] [ 57%] 2024-08-08T21:00:15.5808275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0221s] [ 57%] 2024-08-08T21:00:15.5809805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0571s] [ 57%] 2024-08-08T21:00:15.5811357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0650s] [ 57%] 2024-08-08T21:00:15.5812871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0163s] [ 57%] 2024-08-08T21:00:15.5814465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0166s] [ 57%] 2024-08-08T21:00:15.5816040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0830s] [ 57%] 2024-08-08T21:00:15.5817598Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0849s] [ 58%] 2024-08-08T21:00:15.5819144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0368s] [ 58%] 2024-08-08T21:00:15.5820719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0369s] [ 58%] 2024-08-08T21:00:15.5822243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0393s] [ 58%] 2024-08-08T21:00:15.5823785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0431s] [ 58%] 2024-08-08T21:00:15.5825344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0148s] [ 58%] 2024-08-08T21:00:15.5826945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0149s] [ 58%] 2024-08-08T21:00:15.5828490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0551s] [ 58%] 2024-08-08T21:00:15.5830021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0559s] [ 58%] 2024-08-08T21:00:15.5831565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0268s] [ 58%] 2024-08-08T21:00:15.5833103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0266s] [ 58%] 2024-08-08T21:00:15.5834634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0463s] [ 58%] 2024-08-08T21:00:15.5836155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0510s] [ 58%] 2024-08-08T21:00:15.5837738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0132s] [ 58%] 2024-08-08T21:00:15.5839313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0132s] [ 58%] 2024-08-08T21:00:15.5840829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0752s] [ 58%] 2024-08-08T21:00:15.5842383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0781s] [ 58%] 2024-08-08T21:00:15.5843913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0264s] [ 58%] 2024-08-08T21:00:15.5845460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0265s] [ 58%] 2024-08-08T21:00:15.5846964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0312s] [ 58%] 2024-08-08T21:00:15.5848551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0333s] [ 58%] 2024-08-08T21:00:15.5850116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0122s] [ 58%] 2024-08-08T21:00:15.5851692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0123s] [ 58%] 2024-08-08T21:00:15.5853232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0490s] [ 58%] 2024-08-08T21:00:15.5854790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0501s] [ 58%] 2024-08-08T21:00:15.5856311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0201s] [ 58%] 2024-08-08T21:00:15.5857832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0201s] [ 58%] 2024-08-08T21:00:15.5859759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0109s] [ 58%] 2024-08-08T21:00:15.5861408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0111s] [ 58%] 2024-08-08T21:00:15.5863025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 58%] 2024-08-08T21:00:15.5864548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 58%] 2024-08-08T21:00:15.5866090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0153s] [ 58%] 2024-08-08T21:00:15.5867642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0152s] [ 58%] 2024-08-08T21:00:15.5869189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 58%] 2024-08-08T21:00:15.5870713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 58%] 2024-08-08T21:00:15.5872307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0113s] [ 58%] 2024-08-08T21:00:15.5873911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0115s] [ 58%] 2024-08-08T21:00:15.5875432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 58%] 2024-08-08T21:00:15.5876977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 58%] 2024-08-08T21:00:15.5878504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0156s] [ 58%] 2024-08-08T21:00:15.5880059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0157s] [ 58%] 2024-08-08T21:00:15.5881576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0096s] [ 58%] 2024-08-08T21:00:15.5883119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 58%] 2024-08-08T21:00:15.5884756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0111s] [ 58%] 2024-08-08T21:00:15.5886640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0118s] [ 58%] 2024-08-08T21:00:15.5888748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 58%] 2024-08-08T21:00:15.5890326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 58%] 2024-08-08T21:00:15.5891882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0156s] [ 58%] 2024-08-08T21:00:15.5893423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0156s] [ 58%] 2024-08-08T21:00:15.5894964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0095s] [ 58%] 2024-08-08T21:00:15.5896564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 58%] 2024-08-08T21:00:15.5898198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0116s] [ 58%] 2024-08-08T21:00:15.5899775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0119s] [ 58%] 2024-08-08T21:00:15.5901270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 58%] 2024-08-08T21:00:15.5902800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 58%] 2024-08-08T21:00:15.5904347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0157s] [ 58%] 2024-08-08T21:00:15.5905876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0160s] [ 58%] 2024-08-08T21:00:15.5907393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0096s] [ 58%] 2024-08-08T21:00:15.5908980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 58%] 2024-08-08T21:00:15.5910575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0119s] [ 58%] 2024-08-08T21:00:15.5912080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0123s] [ 58%] 2024-08-08T21:00:15.5913595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0071s] [ 58%] 2024-08-08T21:00:15.5915142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 58%] 2024-08-08T21:00:15.5916666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0163s] [ 58%] 2024-08-08T21:00:15.5918216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0160s] [ 58%] 2024-08-08T21:00:15.5919808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0101s] [ 58%] 2024-08-08T21:00:15.5921399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0102s] [ 58%] 2024-08-08T21:00:15.5922932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0125s] [ 58%] 2024-08-08T21:00:15.5924463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0130s] [ 58%] 2024-08-08T21:00:15.5926000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0076s] [ 58%] 2024-08-08T21:00:15.5927536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 58%] 2024-08-08T21:00:15.5929069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0163s] [ 58%] 2024-08-08T21:00:15.5930594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0165s] [ 58%] 2024-08-08T21:00:15.5932175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0106s] [ 58%] 2024-08-08T21:00:15.5933755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0108s] [ 58%] 2024-08-08T21:00:15.5935282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0107s] [ 58%] 2024-08-08T21:00:15.5936817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0108s] [ 58%] 2024-08-08T21:00:15.5938340Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 59%] 2024-08-08T21:00:15.5939893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 59%] 2024-08-08T21:00:15.5941409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0151s] [ 59%] 2024-08-08T21:00:15.5942995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0150s] [ 59%] 2024-08-08T21:00:15.5944558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 59%] 2024-08-08T21:00:15.5946103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 59%] 2024-08-08T21:00:15.5947605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0109s] [ 59%] 2024-08-08T21:00:15.5949153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 59%] 2024-08-08T21:00:15.5950671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 59%] 2024-08-08T21:00:15.5952212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 59%] 2024-08-08T21:00:15.5953762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0151s] [ 59%] 2024-08-08T21:00:15.5955343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0154s] [ 59%] 2024-08-08T21:00:15.5956929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0094s] [ 59%] 2024-08-08T21:00:15.5958950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0102s] [ 59%] 2024-08-08T21:00:15.5960493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0102s] [ 59%] 2024-08-08T21:00:15.5962068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0104s] [ 59%] 2024-08-08T21:00:15.5963574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0071s] [ 59%] 2024-08-08T21:00:15.5965706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 59%] 2024-08-08T21:00:15.5967428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0144s] [ 59%] 2024-08-08T21:00:15.5969050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0148s] [ 59%] 2024-08-08T21:00:15.5970580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0077s] [ 59%] 2024-08-08T21:00:15.5972125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 59%] 2024-08-08T21:00:15.5973731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0104s] [ 59%] 2024-08-08T21:00:15.5975198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 59%] 2024-08-08T21:00:15.5976761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 59%] 2024-08-08T21:00:15.5978253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 59%] 2024-08-08T21:00:15.5979839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0146s] [ 59%] 2024-08-08T21:00:15.5981465Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0145s] [ 59%] 2024-08-08T21:00:15.5982989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 59%] 2024-08-08T21:00:15.5984517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0083s] [ 59%] 2024-08-08T21:00:15.5986059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0105s] [ 59%] 2024-08-08T21:00:15.5987595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 59%] 2024-08-08T21:00:15.5989131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 59%] 2024-08-08T21:00:15.5990709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 59%] 2024-08-08T21:00:15.5992303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0149s] [ 59%] 2024-08-08T21:00:15.5993838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0148s] [ 59%] 2024-08-08T21:00:15.5995474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 59%] 2024-08-08T21:00:15.5996909Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 59%] 2024-08-08T21:00:15.5998442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0109s] [ 59%] 2024-08-08T21:00:15.5999968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0107s] [ 59%] 2024-08-08T21:00:15.6001477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 59%] 2024-08-08T21:00:15.6003066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 59%] 2024-08-08T21:00:15.6004652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0151s] [ 59%] 2024-08-08T21:00:15.6006220Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0151s] [ 59%] 2024-08-08T21:00:15.6007716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 59%] 2024-08-08T21:00:15.6009278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0083s] [ 59%] 2024-08-08T21:00:15.6010790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0116s] [ 59%] 2024-08-08T21:00:15.6012336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 59%] 2024-08-08T21:00:15.6013900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 59%] 2024-08-08T21:00:15.6015502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 59%] 2024-08-08T21:00:15.6017033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0159s] [ 59%] 2024-08-08T21:00:15.6018568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0162s] [ 59%] 2024-08-08T21:00:15.6020106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 59%] 2024-08-08T21:00:15.6021647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 59%] 2024-08-08T21:00:15.6023181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0120s] [ 59%] 2024-08-08T21:00:15.6024706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0122s] [ 59%] 2024-08-08T21:00:15.6026287Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 59%] 2024-08-08T21:00:15.6027863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 59%] 2024-08-08T21:00:15.6029403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0160s] [ 59%] 2024-08-08T21:00:15.6030932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0160s] [ 59%] 2024-08-08T21:00:15.6032454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 59%] 2024-08-08T21:00:15.6034007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 59%] 2024-08-08T21:00:15.6035523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0099s] [ 59%] 2024-08-08T21:00:15.6037111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0099s] [ 59%] 2024-08-08T21:00:15.6038682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0068s] [ 59%] 2024-08-08T21:00:15.6040215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 59%] 2024-08-08T21:00:15.6041828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0142s] [ 59%] 2024-08-08T21:00:15.6043282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0140s] [ 59%] 2024-08-08T21:00:15.6044852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0078s] [ 59%] 2024-08-08T21:00:15.6046362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 59%] 2024-08-08T21:00:15.6047864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0103s] [ 59%] 2024-08-08T21:00:15.6049528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 59%] 2024-08-08T21:00:15.6051010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0071s] [ 59%] 2024-08-08T21:00:15.6052601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 59%] 2024-08-08T21:00:15.6054109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0143s] [ 59%] 2024-08-08T21:00:15.6055683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0143s] [ 59%] 2024-08-08T21:00:15.6057189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0080s] [ 59%] 2024-08-08T21:00:15.6059074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 59%] 2024-08-08T21:00:15.6060783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 60%] 2024-08-08T21:00:15.6062398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 60%] 2024-08-08T21:00:15.6063919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 60%] 2024-08-08T21:00:15.6065464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 60%] 2024-08-08T21:00:15.6066988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 60%] 2024-08-08T21:00:15.6068580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 60%] 2024-08-08T21:00:15.6070113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 60%] 2024-08-08T21:00:15.6071615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 60%] 2024-08-08T21:00:15.6073190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 60%] 2024-08-08T21:00:15.6074813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 60%] 2024-08-08T21:00:15.6076328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 60%] 2024-08-08T21:00:15.6077922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 60%] 2024-08-08T21:00:15.6079400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 60%] 2024-08-08T21:00:15.6080932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 60%] 2024-08-08T21:00:15.6082569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 60%] 2024-08-08T21:00:15.6084035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 60%] 2024-08-08T21:00:15.6085615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 60%] 2024-08-08T21:00:15.6087147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 60%] 2024-08-08T21:00:15.6088672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 60%] 2024-08-08T21:00:15.6090192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 60%] 2024-08-08T21:00:15.6091744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 60%] 2024-08-08T21:00:15.6093282Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 60%] 2024-08-08T21:00:15.6094797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 60%] 2024-08-08T21:00:15.6096412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 60%] 2024-08-08T21:00:15.6097975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 60%] 2024-08-08T21:00:15.6099540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 60%] 2024-08-08T21:00:15.6101037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 60%] 2024-08-08T21:00:15.6102590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 60%] 2024-08-08T21:00:15.6104116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 60%] 2024-08-08T21:00:15.6105660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 60%] 2024-08-08T21:00:15.6107214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 60%] 2024-08-08T21:00:15.6108893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 60%] 2024-08-08T21:00:15.6110331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0068s] [ 60%] 2024-08-08T21:00:15.6111851Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 60%] 2024-08-08T21:00:15.6113379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 60%] 2024-08-08T21:00:15.6114912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 60%] 2024-08-08T21:00:15.6116456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 60%] 2024-08-08T21:00:15.6117984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 60%] 2024-08-08T21:00:15.6119563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 60%] 2024-08-08T21:00:15.6121151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 60%] 2024-08-08T21:00:15.6122677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 60%] 2024-08-08T21:00:15.6124247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 60%] 2024-08-08T21:00:15.6125723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 60%] 2024-08-08T21:00:15.6127274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 60%] 2024-08-08T21:00:15.6128792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 60%] 2024-08-08T21:00:15.6130377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 60%] 2024-08-08T21:00:15.6131941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 60%] 2024-08-08T21:00:15.6133546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 60%] 2024-08-08T21:00:15.6135112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 60%] 2024-08-08T21:00:15.6136564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 60%] 2024-08-08T21:00:15.6138085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 60%] 2024-08-08T21:00:15.6139633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 60%] 2024-08-08T21:00:15.6141152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 60%] 2024-08-08T21:00:15.6142759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 60%] 2024-08-08T21:00:15.6144339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 60%] 2024-08-08T21:00:15.6145888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 60%] 2024-08-08T21:00:15.6147359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 60%] 2024-08-08T21:00:15.6148890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 60%] 2024-08-08T21:00:15.6150423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 60%] 2024-08-08T21:00:15.6151939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 60%] 2024-08-08T21:00:15.6153549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 60%] 2024-08-08T21:00:15.6155137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 60%] 2024-08-08T21:00:15.6156653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 60%] 2024-08-08T21:00:15.6158256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 60%] 2024-08-08T21:00:15.6160163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0081s] [ 60%] 2024-08-08T21:00:15.6161722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0085s] [ 60%] 2024-08-08T21:00:15.6163250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 60%] 2024-08-08T21:00:15.6164777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 60%] 2024-08-08T21:00:15.6166300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0108s] [ 60%] 2024-08-08T21:00:15.6167945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 60%] 2024-08-08T21:00:15.6169535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 60%] 2024-08-08T21:00:15.6171089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 60%] 2024-08-08T21:00:15.6172607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 60%] 2024-08-08T21:00:15.6174207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 60%] 2024-08-08T21:00:15.6175686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 60%] 2024-08-08T21:00:15.6177274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 60%] 2024-08-08T21:00:15.6178899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 60%] 2024-08-08T21:00:15.6180435Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 60%] 2024-08-08T21:00:15.6181961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 61%] 2024-08-08T21:00:15.6183496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 61%] 2024-08-08T21:00:15.6185035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0083s] [ 61%] 2024-08-08T21:00:15.6186568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0088s] [ 61%] 2024-08-08T21:00:15.6188102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 61%] 2024-08-08T21:00:15.6189626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 61%] 2024-08-08T21:00:15.6191202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0106s] [ 61%] 2024-08-08T21:00:15.6192798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 61%] 2024-08-08T21:00:15.6194317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 61%] 2024-08-08T21:00:15.6195873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 61%] 2024-08-08T21:00:15.6197408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 61%] 2024-08-08T21:00:15.6198944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 61%] 2024-08-08T21:00:15.6200501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 61%] 2024-08-08T21:00:15.6202099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 61%] 2024-08-08T21:00:15.6203627Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 61%] 2024-08-08T21:00:15.6205156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 61%] 2024-08-08T21:00:15.6206688Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 61%] 2024-08-08T21:00:15.6208228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 61%] 2024-08-08T21:00:15.6209762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0091s] [ 61%] 2024-08-08T21:00:15.6211286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 61%] 2024-08-08T21:00:15.6212816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 61%] 2024-08-08T21:00:15.6214385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 61%] 2024-08-08T21:00:15.6215976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0112s] [ 61%] 2024-08-08T21:00:15.6217505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 61%] 2024-08-08T21:00:15.6219077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 61%] 2024-08-08T21:00:15.6220603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 61%] 2024-08-08T21:00:15.6222116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0067s] [ 61%] 2024-08-08T21:00:15.6223711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 61%] 2024-08-08T21:00:15.6225281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 61%] 2024-08-08T21:00:15.6226829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 61%] 2024-08-08T21:00:15.6228342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 61%] 2024-08-08T21:00:15.6229892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 61%] 2024-08-08T21:00:15.6231418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 61%] 2024-08-08T21:00:15.6232969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 61%] 2024-08-08T21:00:15.6234478Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0081s] [ 61%] 2024-08-08T21:00:15.6236029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 61%] 2024-08-08T21:00:15.6237604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 61%] 2024-08-08T21:00:15.6239179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 61%] 2024-08-08T21:00:15.6240721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0102s] [ 61%] 2024-08-08T21:00:15.6242260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0104s] [ 61%] 2024-08-08T21:00:15.6243804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 61%] 2024-08-08T21:00:15.6245336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 61%] 2024-08-08T21:00:15.6246860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 61%] 2024-08-08T21:00:15.6248432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 61%] 2024-08-08T21:00:15.6250017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 61%] 2024-08-08T21:00:15.6251536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 61%] 2024-08-08T21:00:15.6253052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 61%] 2024-08-08T21:00:15.6254651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 61%] 2024-08-08T21:00:15.6256177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 61%] 2024-08-08T21:00:15.6257726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 61%] 2024-08-08T21:00:15.6259648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 61%] 2024-08-08T21:00:15.6261292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 61%] 2024-08-08T21:00:15.6262988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 61%] 2024-08-08T21:00:15.6265935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 61%] 2024-08-08T21:00:15.6268969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 61%] 2024-08-08T21:00:15.6271946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 61%] 2024-08-08T21:00:15.6274880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 61%] 2024-08-08T21:00:15.6277865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 61%] 2024-08-08T21:00:15.6281010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 61%] 2024-08-08T21:00:15.6284021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 61%] 2024-08-08T21:00:15.6287072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 61%] 2024-08-08T21:00:15.6289999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 61%] 2024-08-08T21:00:15.6292981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 61%] 2024-08-08T21:00:15.6295940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 61%] 2024-08-08T21:00:15.6299031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 61%] 2024-08-08T21:00:15.6302004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 61%] 2024-08-08T21:00:15.6305017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 61%] 2024-08-08T21:00:15.6311779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 61%] 2024-08-08T21:00:15.6319493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 61%] 2024-08-08T21:00:15.6327218Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 61%] 2024-08-08T21:00:15.6334980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 61%] 2024-08-08T21:00:15.6342769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 61%] 2024-08-08T21:00:15.6350556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 61%] 2024-08-08T21:00:15.6358718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 61%] 2024-08-08T21:00:15.6366763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 61%] 2024-08-08T21:00:15.6374467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 61%] 2024-08-08T21:00:15.6382185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 61%] 2024-08-08T21:00:15.6389906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 61%] 2024-08-08T21:00:15.6397630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 62%] 2024-08-08T21:00:15.6405347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 62%] 2024-08-08T21:00:15.6413090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 62%] 2024-08-08T21:00:15.6420966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 62%] 2024-08-08T21:00:15.6428810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 62%] 2024-08-08T21:00:15.6436539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 62%] 2024-08-08T21:00:15.6444283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 62%] 2024-08-08T21:00:15.6452051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 62%] 2024-08-08T21:00:15.6460130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 62%] 2024-08-08T21:00:15.6467959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 62%] 2024-08-08T21:00:15.6475874Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 62%] 2024-08-08T21:00:15.6483792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 62%] 2024-08-08T21:00:15.6491448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 62%] 2024-08-08T21:00:15.6499166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 62%] 2024-08-08T21:00:15.6506853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 62%] 2024-08-08T21:00:15.6514580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 62%] 2024-08-08T21:00:15.6522300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 62%] 2024-08-08T21:00:15.6530057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 62%] 2024-08-08T21:00:15.6537929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 62%] 2024-08-08T21:00:15.6545760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 62%] 2024-08-08T21:00:15.6553444Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 62%] 2024-08-08T21:00:15.6561436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 62%] 2024-08-08T21:00:15.6569199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 62%] 2024-08-08T21:00:15.6576932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 62%] 2024-08-08T21:00:15.6584660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 62%] 2024-08-08T21:00:15.6592550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 62%] 2024-08-08T21:00:15.6600453Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 62%] 2024-08-08T21:00:15.6608211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 62%] 2024-08-08T21:00:15.6615884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 62%] 2024-08-08T21:00:15.6623595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 62%] 2024-08-08T21:00:15.6631281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 62%] 2024-08-08T21:00:15.6639011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 62%] 2024-08-08T21:00:15.6646727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 62%] 2024-08-08T21:00:15.6654613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 62%] 2024-08-08T21:00:15.6662748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 62%] 2024-08-08T21:00:15.6670484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 62%] 2024-08-08T21:00:15.6678236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0123s] [ 62%] 2024-08-08T21:00:15.6686021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0129s] [ 62%] 2024-08-08T21:00:15.6693826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 62%] 2024-08-08T21:00:15.6701546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 62%] 2024-08-08T21:00:15.6709514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0165s] [ 62%] 2024-08-08T21:00:15.6717463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0170s] [ 62%] 2024-08-08T21:00:15.6725295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 62%] 2024-08-08T21:00:15.6733119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0089s] [ 62%] 2024-08-08T21:00:15.6740921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0074s] [ 62%] 2024-08-08T21:00:15.6748666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 62%] 2024-08-08T21:00:15.6756437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 62%] 2024-08-08T21:00:15.6764446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 62%] 2024-08-08T21:00:15.6772401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0084s] [ 62%] 2024-08-08T21:00:15.6780353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 62%] 2024-08-08T21:00:15.6788148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 62%] 2024-08-08T21:00:15.6795905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 62%] 2024-08-08T21:00:15.6803750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0130s] [ 62%] 2024-08-08T21:00:15.6811550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0138s] [ 62%] 2024-08-08T21:00:15.6819353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 62%] 2024-08-08T21:00:15.6827281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 62%] 2024-08-08T21:00:15.6835191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0167s] [ 62%] 2024-08-08T21:00:15.6843060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0172s] [ 62%] 2024-08-08T21:00:15.6850899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0089s] [ 62%] 2024-08-08T21:00:15.6858957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 62%] 2024-08-08T21:00:15.6866812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0078s] [ 62%] 2024-08-08T21:00:15.6874573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 62%] 2024-08-08T21:00:15.6882283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 62%] 2024-08-08T21:00:15.6890210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 62%] 2024-08-08T21:00:15.6898121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0087s] [ 62%] 2024-08-08T21:00:15.6905921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0088s] [ 62%] 2024-08-08T21:00:15.6913674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 62%] 2024-08-08T21:00:15.6921462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 62%] 2024-08-08T21:00:15.6929235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0141s] [ 62%] 2024-08-08T21:00:15.6937014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0144s] [ 62%] 2024-08-08T21:00:15.6944973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 62%] 2024-08-08T21:00:15.6952875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 62%] 2024-08-08T21:00:15.6961026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0184s] [ 62%] 2024-08-08T21:00:15.6968822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0188s] [ 62%] 2024-08-08T21:00:15.6976696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0100s] [ 62%] 2024-08-08T21:00:15.6984545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 62%] 2024-08-08T21:00:15.6992316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 62%] 2024-08-08T21:00:15.7000087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 62%] 2024-08-08T21:00:15.7008028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 63%] 2024-08-08T21:00:15.7015931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 63%] 2024-08-08T21:00:15.7023746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0088s] [ 63%] 2024-08-08T21:00:15.7031559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0087s] [ 63%] 2024-08-08T21:00:15.7039387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0080s] [ 63%] 2024-08-08T21:00:15.7047167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 63%] 2024-08-08T21:00:15.7054986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0121s] [ 63%] 2024-08-08T21:00:15.7063175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0127s] [ 63%] 2024-08-08T21:00:15.7071090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 63%] 2024-08-08T21:00:15.7078849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 63%] 2024-08-08T21:00:15.7086592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0162s] [ 63%] 2024-08-08T21:00:15.7094400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0166s] [ 63%] 2024-08-08T21:00:15.7102177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 63%] 2024-08-08T21:00:15.7109960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 63%] 2024-08-08T21:00:15.7117728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0073s] [ 63%] 2024-08-08T21:00:15.7125602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 63%] 2024-08-08T21:00:15.7133418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 63%] 2024-08-08T21:00:15.7141118Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 63%] 2024-08-08T21:00:15.7148848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0083s] [ 63%] 2024-08-08T21:00:15.7156673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 63%] 2024-08-08T21:00:15.7164704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 63%] 2024-08-08T21:00:15.7172479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 63%] 2024-08-08T21:00:15.7180404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 63%] 2024-08-08T21:00:15.7188297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 63%] 2024-08-08T21:00:15.7196089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 63%] 2024-08-08T21:00:15.7203877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 63%] 2024-08-08T21:00:15.7211653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 63%] 2024-08-08T21:00:15.7219433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 63%] 2024-08-08T21:00:15.7227183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 63%] 2024-08-08T21:00:15.7234970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 63%] 2024-08-08T21:00:15.7242868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 63%] 2024-08-08T21:00:15.7250761Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 63%] 2024-08-08T21:00:15.7258746Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 63%] 2024-08-08T21:00:15.7266605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 63%] 2024-08-08T21:00:15.7274333Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 63%] 2024-08-08T21:00:15.7282102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 63%] 2024-08-08T21:00:15.7289867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 63%] 2024-08-08T21:00:15.7297750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 63%] 2024-08-08T21:00:15.7305647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 63%] 2024-08-08T21:00:15.7313425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 63%] 2024-08-08T21:00:15.7321169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 63%] 2024-08-08T21:00:15.7328929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 63%] 2024-08-08T21:00:15.7336724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 63%] 2024-08-08T21:00:15.7344530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 63%] 2024-08-08T21:00:15.7352288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 63%] 2024-08-08T21:00:15.7360443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 63%] 2024-08-08T21:00:15.7368403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 63%] 2024-08-08T21:00:15.7376121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 63%] 2024-08-08T21:00:15.7383831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 63%] 2024-08-08T21:00:15.7391585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 63%] 2024-08-08T21:00:15.7399363Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 63%] 2024-08-08T21:00:15.7407116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 63%] 2024-08-08T21:00:15.7415043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 63%] 2024-08-08T21:00:15.7422946Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 63%] 2024-08-08T21:00:15.7430720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 63%] 2024-08-08T21:00:15.7438484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 63%] 2024-08-08T21:00:15.7446236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 63%] 2024-08-08T21:00:15.7454053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 63%] 2024-08-08T21:00:15.7462114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 63%] 2024-08-08T21:00:15.7469888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 63%] 2024-08-08T21:00:15.7475373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 63%] 2024-08-08T21:00:15.7478451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 63%] 2024-08-08T21:00:15.7481415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 63%] 2024-08-08T21:00:15.7484394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 63%] 2024-08-08T21:00:15.7487370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 63%] 2024-08-08T21:00:15.7490354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 63%] 2024-08-08T21:00:15.7493315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 63%] 2024-08-08T21:00:15.7496343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 63%] 2024-08-08T21:00:15.7499412Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 63%] 2024-08-08T21:00:15.7502402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 63%] 2024-08-08T21:00:15.7505381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 63%] 2024-08-08T21:00:15.7508361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 63%] 2024-08-08T21:00:15.7511352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 63%] 2024-08-08T21:00:15.7514308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 63%] 2024-08-08T21:00:15.7517179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 63%] 2024-08-08T21:00:15.7520227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 63%] 2024-08-08T21:00:15.7523278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 63%] 2024-08-08T21:00:15.7526264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 63%] 2024-08-08T21:00:15.7529207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 64%] 2024-08-08T21:00:15.7532173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 64%] 2024-08-08T21:00:15.7535153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 64%] 2024-08-08T21:00:15.7538114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 64%] 2024-08-08T21:00:15.7541126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 64%] 2024-08-08T21:00:15.7544156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 64%] 2024-08-08T21:00:15.7547104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 64%] 2024-08-08T21:00:15.7550075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 64%] 2024-08-08T21:00:15.7553047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 64%] 2024-08-08T21:00:15.7556094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 64%] 2024-08-08T21:00:15.7559485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 64%] 2024-08-08T21:00:15.7562481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7565462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7568547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 64%] 2024-08-08T21:00:15.7571603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 64%] 2024-08-08T21:00:15.7574591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 64%] 2024-08-08T21:00:15.7577561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 64%] 2024-08-08T21:00:15.7580516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 64%] 2024-08-08T21:00:15.7583466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7586430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 64%] 2024-08-08T21:00:15.7589463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7592512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 64%] 2024-08-08T21:00:15.7595476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7598428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7601404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 64%] 2024-08-08T21:00:15.7604383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 64%] 2024-08-08T21:00:15.7607343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7610322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 64%] 2024-08-08T21:00:15.7613356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 64%] 2024-08-08T21:00:15.7616410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 64%] 2024-08-08T21:00:15.7619387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 64%] 2024-08-08T21:00:15.7622374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 64%] 2024-08-08T21:00:15.7625346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 64%] 2024-08-08T21:00:15.7628304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 64%] 2024-08-08T21:00:15.7631241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 64%] 2024-08-08T21:00:15.7634255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 64%] 2024-08-08T21:00:15.7637273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 64%] 2024-08-08T21:00:15.7640244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7643227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 64%] 2024-08-08T21:00:15.7646178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 64%] 2024-08-08T21:00:15.7649147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7652114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7655124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 64%] 2024-08-08T21:00:15.7658155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 64%] 2024-08-08T21:00:15.7661600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 64%] 2024-08-08T21:00:15.7664560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7667544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 64%] 2024-08-08T21:00:15.7670541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 64%] 2024-08-08T21:00:15.7673516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 64%] 2024-08-08T21:00:15.7676489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 64%] 2024-08-08T21:00:15.7679536Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7682591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 64%] 2024-08-08T21:00:15.7685564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7688532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7691508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 64%] 2024-08-08T21:00:15.7694484Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 64%] 2024-08-08T21:00:15.7697438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 64%] 2024-08-08T21:00:15.7700417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 64%] 2024-08-08T21:00:15.7703450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 64%] 2024-08-08T21:00:15.7706542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 64%] 2024-08-08T21:00:15.7709504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 64%] 2024-08-08T21:00:15.7712468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 64%] 2024-08-08T21:00:15.7715449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 64%] 2024-08-08T21:00:15.7718420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 64%] 2024-08-08T21:00:15.7721357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 64%] 2024-08-08T21:00:15.7724369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 64%] 2024-08-08T21:00:15.7727377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7730304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 64%] 2024-08-08T21:00:15.7733277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7736223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7739194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 64%] 2024-08-08T21:00:15.7742151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 64%] 2024-08-08T21:00:15.7745101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7748145Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7751193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 64%] 2024-08-08T21:00:15.7754215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 64%] 2024-08-08T21:00:15.7757210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 64%] 2024-08-08T21:00:15.7760604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 65%] 2024-08-08T21:00:15.7763588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 65%] 2024-08-08T21:00:15.7766578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 65%] 2024-08-08T21:00:15.7769674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 65%] 2024-08-08T21:00:15.7772742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 65%] 2024-08-08T21:00:15.7775716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 65%] 2024-08-08T21:00:15.7778667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 65%] 2024-08-08T21:00:15.7781646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 65%] 2024-08-08T21:00:15.7784646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 65%] 2024-08-08T21:00:15.7787625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 65%] 2024-08-08T21:00:15.7790596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 65%] 2024-08-08T21:00:15.7793629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0052s] [ 65%] 2024-08-08T21:00:15.7796667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 65%] 2024-08-08T21:00:15.7799660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 65%] 2024-08-08T21:00:15.7802645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 65%] 2024-08-08T21:00:15.7805626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 65%] 2024-08-08T21:00:15.7808628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 65%] 2024-08-08T21:00:15.7811619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 65%] 2024-08-08T21:00:15.7814625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 65%] 2024-08-08T21:00:15.7817656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 65%] 2024-08-08T21:00:15.7820643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 65%] 2024-08-08T21:00:15.7823609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 65%] 2024-08-08T21:00:15.7826585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 65%] 2024-08-08T21:00:15.7829546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 65%] 2024-08-08T21:00:15.7832533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 65%] 2024-08-08T21:00:15.7835511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 65%] 2024-08-08T21:00:15.7838545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 65%] 2024-08-08T21:00:15.7841597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 65%] 2024-08-08T21:00:15.7844575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 65%] 2024-08-08T21:00:15.7847532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 65%] 2024-08-08T21:00:15.7850525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 65%] 2024-08-08T21:00:15.7853497Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 65%] 2024-08-08T21:00:15.7856528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7859980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 65%] 2024-08-08T21:00:15.7863064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7866041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 65%] 2024-08-08T21:00:15.7869007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 65%] 2024-08-08T21:00:15.7871999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 65%] 2024-08-08T21:00:15.7874955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 65%] 2024-08-08T21:00:15.7877931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7880878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 65%] 2024-08-08T21:00:15.7883927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7886987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7889956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 65%] 2024-08-08T21:00:15.7892988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 65%] 2024-08-08T21:00:15.7895923Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 65%] 2024-08-08T21:00:15.7898896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 65%] 2024-08-08T21:00:15.7901870Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 65%] 2024-08-08T21:00:15.7904910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 65%] 2024-08-08T21:00:15.7907941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 65%] 2024-08-08T21:00:15.7910932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 65%] 2024-08-08T21:00:15.7913868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 65%] 2024-08-08T21:00:15.7916835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 65%] 2024-08-08T21:00:15.7919807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 65%] 2024-08-08T21:00:15.7922771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 65%] 2024-08-08T21:00:15.7925741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7928753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 65%] 2024-08-08T21:00:15.7931764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7934728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 65%] 2024-08-08T21:00:15.7937717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 65%] 2024-08-08T21:00:15.7940692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 65%] 2024-08-08T21:00:15.7943686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 65%] 2024-08-08T21:00:15.7946647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 65%] 2024-08-08T21:00:15.7949679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 65%] 2024-08-08T21:00:15.7952726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 65%] 2024-08-08T21:00:15.7955770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 65%] 2024-08-08T21:00:15.7959156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 65%] 2024-08-08T21:00:15.7962143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 65%] 2024-08-08T21:00:15.7965110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 65%] 2024-08-08T21:00:15.7968085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 65%] 2024-08-08T21:00:15.7971052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 65%] 2024-08-08T21:00:15.7974114Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7977182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7980135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 65%] 2024-08-08T21:00:15.7983117Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 65%] 2024-08-08T21:00:15.7986094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0064s] [ 65%] 2024-08-08T21:00:15.7989090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 65%] 2024-08-08T21:00:15.7992060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 65%] 2024-08-08T21:00:15.7995108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 66%] 2024-08-08T21:00:15.7998142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 66%] 2024-08-08T21:00:15.8001137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 66%] 2024-08-08T21:00:15.8004125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 66%] 2024-08-08T21:00:15.8007101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 66%] 2024-08-08T21:00:15.8010088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 66%] 2024-08-08T21:00:15.8013045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 66%] 2024-08-08T21:00:15.8016010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 66%] 2024-08-08T21:00:15.8019046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 66%] 2024-08-08T21:00:15.8022072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 66%] 2024-08-08T21:00:15.8025048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 66%] 2024-08-08T21:00:15.8028041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 66%] 2024-08-08T21:00:15.8030984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 66%] 2024-08-08T21:00:15.8033965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0071s] [ 66%] 2024-08-08T21:00:15.8036939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 66%] 2024-08-08T21:00:15.8039968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 66%] 2024-08-08T21:00:15.8043006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 66%] 2024-08-08T21:00:15.8045969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 66%] 2024-08-08T21:00:15.8048954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 66%] 2024-08-08T21:00:15.8051948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 66%] 2024-08-08T21:00:15.8054985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 66%] 2024-08-08T21:00:15.8057965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 66%] 2024-08-08T21:00:15.8061328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 66%] 2024-08-08T21:00:15.8064294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 66%] 2024-08-08T21:00:15.8067361Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 66%] 2024-08-08T21:00:15.8070402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 66%] 2024-08-08T21:00:15.8073382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 66%] 2024-08-08T21:00:15.8076367Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 66%] 2024-08-08T21:00:15.8079347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 66%] 2024-08-08T21:00:15.8082315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 66%] 2024-08-08T21:00:15.8085297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 66%] 2024-08-08T21:00:15.8088348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 66%] 2024-08-08T21:00:15.8091642Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 66%] 2024-08-08T21:00:15.8094619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 66%] 2024-08-08T21:00:15.8097595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 66%] 2024-08-08T21:00:15.8100587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 66%] 2024-08-08T21:00:15.8103577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 66%] 2024-08-08T21:00:15.8106543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 66%] 2024-08-08T21:00:15.8109513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 66%] 2024-08-08T21:00:15.8112540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 66%] 2024-08-08T21:00:15.8115532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 66%] 2024-08-08T21:00:15.8118499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 66%] 2024-08-08T21:00:15.8121483Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 66%] 2024-08-08T21:00:15.8124455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 66%] 2024-08-08T21:00:15.8127430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 66%] 2024-08-08T21:00:15.8130358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 66%] 2024-08-08T21:00:15.8133386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 66%] 2024-08-08T21:00:15.8136418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 66%] 2024-08-08T21:00:15.8139379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 66%] 2024-08-08T21:00:15.8142345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 66%] 2024-08-08T21:00:15.8145334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 66%] 2024-08-08T21:00:15.8148288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 66%] 2024-08-08T21:00:15.8151248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 66%] 2024-08-08T21:00:15.8154245Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 66%] 2024-08-08T21:00:15.8157264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 66%] 2024-08-08T21:00:15.8160696Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 66%] 2024-08-08T21:00:15.8163654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 66%] 2024-08-08T21:00:15.8166591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 66%] 2024-08-08T21:00:15.8169559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 66%] 2024-08-08T21:00:15.8172508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 66%] 2024-08-08T21:00:15.8175470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 66%] 2024-08-08T21:00:15.8178521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0052s] [ 66%] 2024-08-08T21:00:15.8181571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 66%] 2024-08-08T21:00:15.8184538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 66%] 2024-08-08T21:00:15.8187503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 66%] 2024-08-08T21:00:15.8190479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 66%] 2024-08-08T21:00:15.8193464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 66%] 2024-08-08T21:00:15.8196407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 66%] 2024-08-08T21:00:15.8199384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 66%] 2024-08-08T21:00:15.8202407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 66%] 2024-08-08T21:00:15.8205420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 66%] 2024-08-08T21:00:15.8208368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 66%] 2024-08-08T21:00:15.8211315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 66%] 2024-08-08T21:00:15.8214261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 66%] 2024-08-08T21:00:15.8217223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 66%] 2024-08-08T21:00:15.8220176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 66%] 2024-08-08T21:00:15.8223182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 66%] 2024-08-08T21:00:15.8226203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0052s] [ 66%] 2024-08-08T21:00:15.8229151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 67%] 2024-08-08T21:00:15.8232112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 67%] 2024-08-08T21:00:15.8235075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 67%] 2024-08-08T21:00:15.8238044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 67%] 2024-08-08T21:00:15.8241015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 67%] 2024-08-08T21:00:15.8243986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 67%] 2024-08-08T21:00:15.8246996Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 67%] 2024-08-08T21:00:15.8249998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 67%] 2024-08-08T21:00:15.8252938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 67%] 2024-08-08T21:00:15.8255942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 67%] 2024-08-08T21:00:15.8259276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 67%] 2024-08-08T21:00:15.8262254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 67%] 2024-08-08T21:00:15.8265216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 67%] 2024-08-08T21:00:15.8268260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8271295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8274254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 67%] 2024-08-08T21:00:15.8277201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 67%] 2024-08-08T21:00:15.8280135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 67%] 2024-08-08T21:00:15.8283100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 67%] 2024-08-08T21:00:15.8286063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 67%] 2024-08-08T21:00:15.8289026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 67%] 2024-08-08T21:00:15.8292069Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 67%] 2024-08-08T21:00:15.8295081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 67%] 2024-08-08T21:00:15.8298030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 67%] 2024-08-08T21:00:15.8300964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 67%] 2024-08-08T21:00:15.8303906Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 67%] 2024-08-08T21:00:15.8306860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 67%] 2024-08-08T21:00:15.8309798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 67%] 2024-08-08T21:00:15.8312787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 67%] 2024-08-08T21:00:15.8315789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8318750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 67%] 2024-08-08T21:00:15.8321715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0083s] [ 67%] 2024-08-08T21:00:15.8324715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 67%] 2024-08-08T21:00:15.8327689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 67%] 2024-08-08T21:00:15.8330682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 67%] 2024-08-08T21:00:15.8333697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 67%] 2024-08-08T21:00:15.8336730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0095s] [ 67%] 2024-08-08T21:00:15.8339777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 67%] 2024-08-08T21:00:15.8342789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 67%] 2024-08-08T21:00:15.8345724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 67%] 2024-08-08T21:00:15.8348702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 67%] 2024-08-08T21:00:15.8351685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 67%] 2024-08-08T21:00:15.8354700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 67%] 2024-08-08T21:00:15.8357747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8361161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 67%] 2024-08-08T21:00:15.8364152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8367140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8370122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 67%] 2024-08-08T21:00:15.8373110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 67%] 2024-08-08T21:00:15.8376089Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 67%] 2024-08-08T21:00:15.8379065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 67%] 2024-08-08T21:00:15.8382056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0102s] [ 67%] 2024-08-08T21:00:15.8385146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 67%] 2024-08-08T21:00:15.8388208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 67%] 2024-08-08T21:00:15.8391197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 67%] 2024-08-08T21:00:15.8394160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 67%] 2024-08-08T21:00:15.8397134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 67%] 2024-08-08T21:00:15.8400101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 67%] 2024-08-08T21:00:15.8403065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 67%] 2024-08-08T21:00:15.8406104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 67%] 2024-08-08T21:00:15.8409166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 67%] 2024-08-08T21:00:15.8412120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 67%] 2024-08-08T21:00:15.8415100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 67%] 2024-08-08T21:00:15.8418084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0108s] [ 67%] 2024-08-08T21:00:15.8421074Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0109s] [ 67%] 2024-08-08T21:00:15.8424047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 67%] 2024-08-08T21:00:15.8427034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 67%] 2024-08-08T21:00:15.8430100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0117s] [ 67%] 2024-08-08T21:00:15.8433142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0118s] [ 67%] 2024-08-08T21:00:15.8436135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8439126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8442119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 67%] 2024-08-08T21:00:15.8445067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 67%] 2024-08-08T21:00:15.8448037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 67%] 2024-08-08T21:00:15.8451051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8454110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 67%] 2024-08-08T21:00:15.8457104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 67%] 2024-08-08T21:00:15.8460508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 67%] 2024-08-08T21:00:15.8463490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 68%] 2024-08-08T21:00:15.8466459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 68%] 2024-08-08T21:00:15.8469448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 68%] 2024-08-08T21:00:15.8472394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 68%] 2024-08-08T21:00:15.8475462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 68%] 2024-08-08T21:00:15.8478490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0089s] [ 68%] 2024-08-08T21:00:15.8481476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 68%] 2024-08-08T21:00:15.8484462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 68%] 2024-08-08T21:00:15.8487463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 68%] 2024-08-08T21:00:15.8489044Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 68%] 2024-08-08T21:00:15.8490509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 68%] 2024-08-08T21:00:15.8492086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 68%] 2024-08-08T21:00:15.8493708Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 68%] 2024-08-08T21:00:15.8495224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 68%] 2024-08-08T21:00:15.8496771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 68%] 2024-08-08T21:00:15.8498293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 68%] 2024-08-08T21:00:15.8499840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 68%] 2024-08-08T21:00:15.8501350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0052s] [ 68%] 2024-08-08T21:00:15.8502883Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 68%] 2024-08-08T21:00:15.8504466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 68%] 2024-08-08T21:00:15.8506063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 68%] 2024-08-08T21:00:15.8507597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 68%] 2024-08-08T21:00:15.8509130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 68%] 2024-08-08T21:00:15.8510668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 68%] 2024-08-08T21:00:15.8512284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 68%] 2024-08-08T21:00:15.8513727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 68%] 2024-08-08T21:00:15.8515294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 68%] 2024-08-08T21:00:15.8516853Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 68%] 2024-08-08T21:00:15.8518391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 68%] 2024-08-08T21:00:15.8519904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 68%] 2024-08-08T21:00:15.8521457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 68%] 2024-08-08T21:00:15.8522979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 68%] 2024-08-08T21:00:15.8524519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 68%] 2024-08-08T21:00:15.8526025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 68%] 2024-08-08T21:00:15.8527646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 68%] 2024-08-08T21:00:15.8529189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 68%] 2024-08-08T21:00:15.8530729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 68%] 2024-08-08T21:00:15.8532250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 68%] 2024-08-08T21:00:15.8533792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 68%] 2024-08-08T21:00:15.8535328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 68%] 2024-08-08T21:00:15.8536855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 68%] 2024-08-08T21:00:15.8538421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 68%] 2024-08-08T21:00:15.8539995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 68%] 2024-08-08T21:00:15.8541516Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 68%] 2024-08-08T21:00:15.8543034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 68%] 2024-08-08T21:00:15.8544570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 68%] 2024-08-08T21:00:15.8546104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 68%] 2024-08-08T21:00:15.8547618Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 68%] 2024-08-08T21:00:15.8549158Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 68%] 2024-08-08T21:00:15.8550723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0052s] [ 68%] 2024-08-08T21:00:15.8552319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 68%] 2024-08-08T21:00:15.8553831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 68%] 2024-08-08T21:00:15.8555418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 68%] 2024-08-08T21:00:15.8556952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 68%] 2024-08-08T21:00:15.8558834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 68%] 2024-08-08T21:00:15.8560397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 68%] 2024-08-08T21:00:15.8562026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 68%] 2024-08-08T21:00:15.8563601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 68%] 2024-08-08T21:00:15.8565120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 68%] 2024-08-08T21:00:15.8566648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 68%] 2024-08-08T21:00:15.8568167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 68%] 2024-08-08T21:00:15.8569720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 68%] 2024-08-08T21:00:15.8571244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 68%] 2024-08-08T21:00:15.8572767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 68%] 2024-08-08T21:00:15.8574349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 68%] 2024-08-08T21:00:15.8575945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 68%] 2024-08-08T21:00:15.8577460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 68%] 2024-08-08T21:00:15.8578965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 68%] 2024-08-08T21:00:15.8580508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 68%] 2024-08-08T21:00:15.8582029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 68%] 2024-08-08T21:00:15.8583569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 68%] 2024-08-08T21:00:15.8585122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 68%] 2024-08-08T21:00:15.8586732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 68%] 2024-08-08T21:00:15.8588216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 68%] 2024-08-08T21:00:15.8589740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 68%] 2024-08-08T21:00:15.8591236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 68%] 2024-08-08T21:00:15.8592760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 68%] 2024-08-08T21:00:15.8594290Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 68%] 2024-08-08T21:00:15.8595803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 69%] 2024-08-08T21:00:15.8597388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 69%] 2024-08-08T21:00:15.8598959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 69%] 2024-08-08T21:00:15.8600481Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8601997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8603527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 69%] 2024-08-08T21:00:15.8605056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 69%] 2024-08-08T21:00:15.8606601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 69%] 2024-08-08T21:00:15.8608175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 69%] 2024-08-08T21:00:15.8609741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 69%] 2024-08-08T21:00:15.8611284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 69%] 2024-08-08T21:00:15.8612782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 69%] 2024-08-08T21:00:15.8614313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 69%] 2024-08-08T21:00:15.8615816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 69%] 2024-08-08T21:00:15.8617351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8618855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 69%] 2024-08-08T21:00:15.8620436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 69%] 2024-08-08T21:00:15.8621993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 69%] 2024-08-08T21:00:15.8623503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 69%] 2024-08-08T21:00:15.8625022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8626541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8628071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 69%] 2024-08-08T21:00:15.8629587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 69%] 2024-08-08T21:00:15.8631162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 69%] 2024-08-08T21:00:15.8632738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 69%] 2024-08-08T21:00:15.8634271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 69%] 2024-08-08T21:00:15.8635792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 69%] 2024-08-08T21:00:15.8637310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 69%] 2024-08-08T21:00:15.8638830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 69%] 2024-08-08T21:00:15.8640329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 69%] 2024-08-08T21:00:15.8641855Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 69%] 2024-08-08T21:00:15.8643400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 69%] 2024-08-08T21:00:15.8644966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 69%] 2024-08-08T21:00:15.8646498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 69%] 2024-08-08T21:00:15.8648027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 69%] 2024-08-08T21:00:15.8649537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8651126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8652582Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 69%] 2024-08-08T21:00:15.8654130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 69%] 2024-08-08T21:00:15.8655713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 69%] 2024-08-08T21:00:15.8657301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 69%] 2024-08-08T21:00:15.8659177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 69%] 2024-08-08T21:00:15.8660738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 69%] 2024-08-08T21:00:15.8662261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 69%] 2024-08-08T21:00:15.8663778Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 69%] 2024-08-08T21:00:15.8665296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 69%] 2024-08-08T21:00:15.8666805Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 69%] 2024-08-08T21:00:15.8668418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 69%] 2024-08-08T21:00:15.8670013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 69%] 2024-08-08T21:00:15.8671517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 69%] 2024-08-08T21:00:15.8673055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 69%] 2024-08-08T21:00:15.8674563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8676099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8677595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 69%] 2024-08-08T21:00:15.8679198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 69%] 2024-08-08T21:00:15.8680783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 69%] 2024-08-08T21:00:15.8691234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 69%] 2024-08-08T21:00:15.8692769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 69%] 2024-08-08T21:00:15.8694331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 69%] 2024-08-08T21:00:15.8695845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 69%] 2024-08-08T21:00:15.8697374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 69%] 2024-08-08T21:00:15.8698871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 69%] 2024-08-08T21:00:15.8700511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 69%] 2024-08-08T21:00:15.8702083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 69%] 2024-08-08T21:00:15.8703625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 69%] 2024-08-08T21:00:15.8705129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 69%] 2024-08-08T21:00:15.8706657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_1_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 69%] 2024-08-08T21:00:15.8708215Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0098s] [ 69%] 2024-08-08T21:00:15.8709758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0100s] [ 69%] 2024-08-08T21:00:15.8711347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 69%] 2024-08-08T21:00:15.8712938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 69%] 2024-08-08T21:00:15.8714496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0126s] [ 69%] 2024-08-08T21:00:15.8716047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0126s] [ 69%] 2024-08-08T21:00:15.8717609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 69%] 2024-08-08T21:00:15.8719168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 69%] 2024-08-08T21:00:15.8720687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0094s] [ 69%] 2024-08-08T21:00:15.8722246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 69%] 2024-08-08T21:00:15.8723823Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 69%] 2024-08-08T21:00:15.8725418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 70%] 2024-08-08T21:00:15.8726944Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0117s] [ 70%] 2024-08-08T21:00:15.8728504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 70%] 2024-08-08T21:00:15.8730037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 70%] 2024-08-08T21:00:15.8731587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 70%] 2024-08-08T21:00:15.8733103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0102s] [ 70%] 2024-08-08T21:00:15.8734693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 70%] 2024-08-08T21:00:15.8736263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 70%] 2024-08-08T21:00:15.8737794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 70%] 2024-08-08T21:00:15.8739341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0131s] [ 70%] 2024-08-08T21:00:15.8740891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0132s] [ 70%] 2024-08-08T21:00:15.8742463Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 70%] 2024-08-08T21:00:15.8744108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0094s] [ 70%] 2024-08-08T21:00:15.8745565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0103s] [ 70%] 2024-08-08T21:00:15.8747148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 70%] 2024-08-08T21:00:15.8748722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 70%] 2024-08-08T21:00:15.8750244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 70%] 2024-08-08T21:00:15.8751785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0123s] [ 70%] 2024-08-08T21:00:15.8753334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0124s] [ 70%] 2024-08-08T21:00:15.8754918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0092s] [ 70%] 2024-08-08T21:00:15.8756474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 70%] 2024-08-08T21:00:15.8758046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0119s] [ 70%] 2024-08-08T21:00:15.8760182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0123s] [ 70%] 2024-08-08T21:00:15.8761742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0088s] [ 70%] 2024-08-08T21:00:15.8763259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 70%] 2024-08-08T21:00:15.8764789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0149s] [ 70%] 2024-08-08T21:00:15.8766358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0149s] [ 70%] 2024-08-08T21:00:15.8767919Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0113s] [ 70%] 2024-08-08T21:00:15.8769442Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 70%] 2024-08-08T21:00:15.8771104Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0115s] [ 70%] 2024-08-08T21:00:15.8772713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 70%] 2024-08-08T21:00:15.8774242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0088s] [ 70%] 2024-08-08T21:00:15.8775770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 70%] 2024-08-08T21:00:15.8777317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0138s] [ 70%] 2024-08-08T21:00:15.8778861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0137s] [ 70%] 2024-08-08T21:00:15.8780393Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0109s] [ 70%] 2024-08-08T21:00:15.8782004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0108s] [ 70%] 2024-08-08T21:00:15.8783612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0094s] [ 70%] 2024-08-08T21:00:15.8785141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 70%] 2024-08-08T21:00:15.8786648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 70%] 2024-08-08T21:00:15.8788191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 70%] 2024-08-08T21:00:15.8789712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0123s] [ 70%] 2024-08-08T21:00:15.8791269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0122s] [ 70%] 2024-08-08T21:00:15.8792785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0086s] [ 70%] 2024-08-08T21:00:15.8794376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 70%] 2024-08-08T21:00:15.8795931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0092s] [ 70%] 2024-08-08T21:00:15.8797474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 70%] 2024-08-08T21:00:15.8798986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0073s] [ 70%] 2024-08-08T21:00:15.8800539Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 70%] 2024-08-08T21:00:15.8802063Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0114s] [ 70%] 2024-08-08T21:00:15.8803590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 70%] 2024-08-08T21:00:15.8805160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0086s] [ 70%] 2024-08-08T21:00:15.8806737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 70%] 2024-08-08T21:00:15.8808273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0158s] [ 70%] 2024-08-08T21:00:15.8809799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0166s] [ 70%] 2024-08-08T21:00:15.8811338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0104s] [ 70%] 2024-08-08T21:00:15.8812873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0105s] [ 70%] 2024-08-08T21:00:15.8814417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0213s] [ 70%] 2024-08-08T21:00:15.8815964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0217s] [ 70%] 2024-08-08T21:00:15.8817548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0132s] [ 70%] 2024-08-08T21:00:15.8819156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 70%] 2024-08-08T21:00:15.8820676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0129s] [ 70%] 2024-08-08T21:00:15.8822224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0132s] [ 70%] 2024-08-08T21:00:15.8823747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0106s] [ 70%] 2024-08-08T21:00:15.8825288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0109s] [ 70%] 2024-08-08T21:00:15.8826802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0156s] [ 70%] 2024-08-08T21:00:15.8828408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0156s] [ 70%] 2024-08-08T21:00:15.8830021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0124s] [ 70%] 2024-08-08T21:00:15.8831565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 70%] 2024-08-08T21:00:15.8833077Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0170s] [ 70%] 2024-08-08T21:00:15.8834605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0181s] [ 70%] 2024-08-08T21:00:15.8836144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0115s] [ 70%] 2024-08-08T21:00:15.8837697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0118s] [ 70%] 2024-08-08T21:00:15.8839219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0224s] [ 70%] 2024-08-08T21:00:15.8840905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0228s] [ 70%] 2024-08-08T21:00:15.8842424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0143s] [ 70%] 2024-08-08T21:00:15.8843968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0143s] [ 70%] 2024-08-08T21:00:15.8845503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0141s] [ 71%] 2024-08-08T21:00:15.8847041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0141s] [ 71%] 2024-08-08T21:00:15.8848557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0116s] [ 71%] 2024-08-08T21:00:15.8850093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0115s] [ 71%] 2024-08-08T21:00:15.8851656Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0170s] [ 71%] 2024-08-08T21:00:15.8853263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0172s] [ 71%] 2024-08-08T21:00:15.8854821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0136s] [ 71%] 2024-08-08T21:00:15.8856370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 71%] 2024-08-08T21:00:15.8857911Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0200s] [ 71%] 2024-08-08T21:00:15.8859833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0212s] [ 71%] 2024-08-08T21:00:15.8861383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.1867s] [ 71%] 2024-08-08T21:00:15.8862938Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0155s] [ 71%] 2024-08-08T21:00:15.8864559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0272s] [ 71%] 2024-08-08T21:00:15.8866193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0275s] [ 71%] 2024-08-08T21:00:15.8867747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0199s] [ 71%] 2024-08-08T21:00:15.8869298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0200s] [ 71%] 2024-08-08T21:00:15.8870847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0179s] [ 71%] 2024-08-08T21:00:15.8872379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0181s] [ 71%] 2024-08-08T21:00:15.8873914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0155s] [ 71%] 2024-08-08T21:00:15.8875501Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0147s] [ 71%] 2024-08-08T21:00:15.8877125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0204s] [ 71%] 2024-08-08T21:00:15.8878663Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0204s] [ 71%] 2024-08-08T21:00:15.8880183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0175s] [ 71%] 2024-08-08T21:00:15.8881722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0182s] [ 71%] 2024-08-08T21:00:15.8883242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0164s] [ 71%] 2024-08-08T21:00:15.8884782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0168s] [ 71%] 2024-08-08T21:00:15.8886298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0113s] [ 71%] 2024-08-08T21:00:15.8887896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0113s] [ 71%] 2024-08-08T21:00:15.8889470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0218s] [ 71%] 2024-08-08T21:00:15.8891028Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0220s] [ 71%] 2024-08-08T21:00:15.8892532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0141s] [ 71%] 2024-08-08T21:00:15.8894088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0140s] [ 71%] 2024-08-08T21:00:15.8895605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0135s] [ 71%] 2024-08-08T21:00:15.8897131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0133s] [ 71%] 2024-08-08T21:00:15.8898712Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0111s] [ 71%] 2024-08-08T21:00:15.8900294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0113s] [ 71%] 2024-08-08T21:00:15.8901833Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0159s] [ 71%] 2024-08-08T21:00:15.8903365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0158s] [ 71%] 2024-08-08T21:00:15.8904903Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0134s] [ 71%] 2024-08-08T21:00:15.8906426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 71%] 2024-08-08T21:00:15.8907957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0067s] [ 71%] 2024-08-08T21:00:15.8909495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 71%] 2024-08-08T21:00:15.8911080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 71%] 2024-08-08T21:00:15.8912671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 71%] 2024-08-08T21:00:15.8914173Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 71%] 2024-08-08T21:00:15.8915730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 71%] 2024-08-08T21:00:15.8917252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 71%] 2024-08-08T21:00:15.8918796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 71%] 2024-08-08T21:00:15.8920299Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 71%] 2024-08-08T21:00:15.8921890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 71%] 2024-08-08T21:00:15.8923455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 71%] 2024-08-08T21:00:15.8924992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 71%] 2024-08-08T21:00:15.8926507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 71%] 2024-08-08T21:00:15.8928046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 71%] 2024-08-08T21:00:15.8929583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 71%] 2024-08-08T21:00:15.8931132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 71%] 2024-08-08T21:00:15.8932643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 71%] 2024-08-08T21:00:15.8934229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 71%] 2024-08-08T21:00:15.8935815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 71%] 2024-08-08T21:00:15.8937339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 71%] 2024-08-08T21:00:15.8938885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0085s] [ 71%] 2024-08-08T21:00:15.8940431Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 71%] 2024-08-08T21:00:15.8941967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 71%] 2024-08-08T21:00:15.8943491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 71%] 2024-08-08T21:00:15.8945039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 71%] 2024-08-08T21:00:15.8946634Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 71%] 2024-08-08T21:00:15.8948150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 71%] 2024-08-08T21:00:15.8949692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 71%] 2024-08-08T21:00:15.8951216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0081s] [ 71%] 2024-08-08T21:00:15.8952783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 71%] 2024-08-08T21:00:15.8954394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 71%] 2024-08-08T21:00:15.8955897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 71%] 2024-08-08T21:00:15.8957475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0076s] [ 71%] 2024-08-08T21:00:15.8959415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 71%] 2024-08-08T21:00:15.8960980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 71%] 2024-08-08T21:00:15.8962514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 71%] 2024-08-08T21:00:15.8964057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0088s] [ 71%] 2024-08-08T21:00:15.8965594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 71%] 2024-08-08T21:00:15.8967127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 72%] 2024-08-08T21:00:15.8968739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 72%] 2024-08-08T21:00:15.8970342Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 72%] 2024-08-08T21:00:15.8971864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 72%] 2024-08-08T21:00:15.8973385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 72%] 2024-08-08T21:00:15.8974912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 72%] 2024-08-08T21:00:15.8976434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0091s] [ 72%] 2024-08-08T21:00:15.8977993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 72%] 2024-08-08T21:00:15.8979511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 72%] 2024-08-08T21:00:15.8981121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 72%] 2024-08-08T21:00:15.8982691Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 72%] 2024-08-08T21:00:15.8984235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 72%] 2024-08-08T21:00:15.8985754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 72%] 2024-08-08T21:00:15.8987303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 72%] 2024-08-08T21:00:15.8988821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 72%] 2024-08-08T21:00:15.8990351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 72%] 2024-08-08T21:00:15.8991927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 72%] 2024-08-08T21:00:15.8993507Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 72%] 2024-08-08T21:00:15.8995031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 72%] 2024-08-08T21:00:15.8996549Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 72%] 2024-08-08T21:00:15.8998082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 72%] 2024-08-08T21:00:15.8999610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 72%] 2024-08-08T21:00:15.9001142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 72%] 2024-08-08T21:00:15.9002666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 72%] 2024-08-08T21:00:15.9004237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 72%] 2024-08-08T21:00:15.9005808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 72%] 2024-08-08T21:00:15.9007338Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0299s] [ 72%] 2024-08-08T21:00:15.9008894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1994s] [ 72%] 2024-08-08T21:00:15.9010420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0180s] [ 72%] 2024-08-08T21:00:15.9011969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0188s] [ 72%] 2024-08-08T21:00:15.9013496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0406s] [ 72%] 2024-08-08T21:00:15.9015111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0420s] [ 72%] 2024-08-08T21:00:15.9016702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0241s] [ 72%] 2024-08-08T21:00:15.9018278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0244s] [ 72%] 2024-08-08T21:00:15.9019770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0212s] [ 72%] 2024-08-08T21:00:15.9021320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0215s] [ 72%] 2024-08-08T21:00:15.9022841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0191s] [ 72%] 2024-08-08T21:00:15.9024364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0190s] [ 72%] 2024-08-08T21:00:15.9025904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0259s] [ 72%] 2024-08-08T21:00:15.9027493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0251s] [ 72%] 2024-08-08T21:00:15.9029102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0217s] [ 72%] 2024-08-08T21:00:15.9030608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0218s] [ 72%] 2024-08-08T21:00:15.9032144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0329s] [ 72%] 2024-08-08T21:00:15.9033680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0348s] [ 72%] 2024-08-08T21:00:15.9035216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0213s] [ 72%] 2024-08-08T21:00:15.9036736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0210s] [ 72%] 2024-08-08T21:00:15.9038305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0436s] [ 72%] 2024-08-08T21:00:15.9039930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0446s] [ 72%] 2024-08-08T21:00:15.9041476Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.1888s] [ 72%] 2024-08-08T21:00:15.9043042Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0270s] [ 72%] 2024-08-08T21:00:15.9044567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0240s] [ 72%] 2024-08-08T21:00:15.9046121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0240s] [ 72%] 2024-08-08T21:00:15.9047629Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0214s] [ 72%] 2024-08-08T21:00:15.9049170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0209s] [ 72%] 2024-08-08T21:00:15.9050738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0278s] [ 72%] 2024-08-08T21:00:15.9052336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0277s] [ 72%] 2024-08-08T21:00:15.9053847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0246s] [ 72%] 2024-08-08T21:00:15.9055420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0245s] [ 72%] 2024-08-08T21:00:15.9056965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0388s] [ 72%] 2024-08-08T21:00:15.9058826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0407s] [ 72%] 2024-08-08T21:00:15.9060439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0261s] [ 72%] 2024-08-08T21:00:15.9062067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0254s] [ 72%] 2024-08-08T21:00:15.9063701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0486s] [ 72%] 2024-08-08T21:00:15.9065246Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0496s] [ 72%] 2024-08-08T21:00:15.9066800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0347s] [ 72%] 2024-08-08T21:00:15.9068371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1987s] [ 72%] 2024-08-08T21:00:15.9069887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0276s] [ 72%] 2024-08-08T21:00:15.9071434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0279s] [ 72%] 2024-08-08T21:00:15.9072950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0253s] [ 72%] 2024-08-08T21:00:15.9074557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0247s] [ 72%] 2024-08-08T21:00:15.9076157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0331s] [ 72%] 2024-08-08T21:00:15.9077707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0326s] [ 72%] 2024-08-08T21:00:15.9079232Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0293s] [ 72%] 2024-08-08T21:00:15.9080785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0285s] [ 72%] 2024-08-08T21:00:15.9082304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0285s] [ 72%] 2024-08-08T21:00:15.9083848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0294s] [ 72%] 2024-08-08T21:00:15.9085407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0173s] [ 72%] 2024-08-08T21:00:15.9086987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0180s] [ 72%] 2024-08-08T21:00:15.9088525Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0391s] [ 73%] 2024-08-08T21:00:15.9090061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0402s] [ 73%] 2024-08-08T21:00:15.9091606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0228s] [ 73%] 2024-08-08T21:00:15.9093143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0233s] [ 73%] 2024-08-08T21:00:15.9094670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0200s] [ 73%] 2024-08-08T21:00:15.9096206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0205s] [ 73%] 2024-08-08T21:00:15.9097789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0184s] [ 73%] 2024-08-08T21:00:15.9099364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.1858s] [ 73%] 2024-08-08T21:00:15.9100900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0242s] [ 73%] 2024-08-08T21:00:15.9102439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0242s] [ 73%] 2024-08-08T21:00:15.9103964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0210s] [ 73%] 2024-08-08T21:00:15.9105506Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0212s] [ 73%] 2024-08-08T21:00:15.9107018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0072s] [ 73%] 2024-08-08T21:00:15.9108605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 73%] 2024-08-08T21:00:15.9110185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 73%] 2024-08-08T21:00:15.9111710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 73%] 2024-08-08T21:00:15.9113225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0096s] [ 73%] 2024-08-08T21:00:15.9114807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0096s] [ 73%] 2024-08-08T21:00:15.9116310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 73%] 2024-08-08T21:00:15.9117836Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 73%] 2024-08-08T21:00:15.9119368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 73%] 2024-08-08T21:00:15.9120952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 73%] 2024-08-08T21:00:15.9122528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 73%] 2024-08-08T21:00:15.9124056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 73%] 2024-08-08T21:00:15.9125600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0096s] [ 73%] 2024-08-08T21:00:15.9127140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 73%] 2024-08-08T21:00:15.9128675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0082s] [ 73%] 2024-08-08T21:00:15.9130200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 73%] 2024-08-08T21:00:15.9131786Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 73%] 2024-08-08T21:00:15.9133365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 73%] 2024-08-08T21:00:15.9134873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 73%] 2024-08-08T21:00:15.9136408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 73%] 2024-08-08T21:00:15.9137936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0101s] [ 73%] 2024-08-08T21:00:15.9139503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0100s] [ 73%] 2024-08-08T21:00:15.9141021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0090s] [ 73%] 2024-08-08T21:00:15.9142572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0089s] [ 73%] 2024-08-08T21:00:15.9144132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0085s] [ 73%] 2024-08-08T21:00:15.9145726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 73%] 2024-08-08T21:00:15.9147239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 73%] 2024-08-08T21:00:15.9148771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 73%] 2024-08-08T21:00:15.9150314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0091s] [ 73%] 2024-08-08T21:00:15.9151849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 73%] 2024-08-08T21:00:15.9153377Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 73%] 2024-08-08T21:00:15.9154994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 73%] 2024-08-08T21:00:15.9156580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0083s] [ 73%] 2024-08-08T21:00:15.9158098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0083s] [ 73%] 2024-08-08T21:00:15.9159976Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0072s] [ 73%] 2024-08-08T21:00:15.9161521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 73%] 2024-08-08T21:00:15.9163068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0102s] [ 73%] 2024-08-08T21:00:15.9164604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0104s] [ 73%] 2024-08-08T21:00:15.9166123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0089s] [ 73%] 2024-08-08T21:00:15.9167760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0089s] [ 73%] 2024-08-08T21:00:15.9169341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0089s] [ 73%] 2024-08-08T21:00:15.9170891Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 73%] 2024-08-08T21:00:15.9172405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0078s] [ 73%] 2024-08-08T21:00:15.9173954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 73%] 2024-08-08T21:00:15.9175477Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0105s] [ 73%] 2024-08-08T21:00:15.9177023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 73%] 2024-08-08T21:00:15.9178594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0091s] [ 73%] 2024-08-08T21:00:15.9180207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 73%] 2024-08-08T21:00:15.9181721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0068s] [ 73%] 2024-08-08T21:00:15.9183250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 73%] 2024-08-08T21:00:15.9184784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 73%] 2024-08-08T21:00:15.9186318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 73%] 2024-08-08T21:00:15.9187860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0081s] [ 73%] 2024-08-08T21:00:15.9189394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0085s] [ 73%] 2024-08-08T21:00:15.9190972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 73%] 2024-08-08T21:00:15.9192543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 73%] 2024-08-08T21:00:15.9194066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0072s] [ 73%] 2024-08-08T21:00:15.9195595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 73%] 2024-08-08T21:00:15.9197108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 73%] 2024-08-08T21:00:15.9198648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 73%] 2024-08-08T21:00:15.9200157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0085s] [ 73%] 2024-08-08T21:00:15.9201799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 73%] 2024-08-08T21:00:15.9203375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0070s] [ 73%] 2024-08-08T21:00:15.9204914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 73%] 2024-08-08T21:00:15.9206428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 73%] 2024-08-08T21:00:15.9207978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 [32mPASSED [0.0062s] [ 73%] 2024-08-08T21:00:15.9209503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 74%] 2024-08-08T21:00:15.9211046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 74%] 2024-08-08T21:00:15.9212565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0079s] [ 74%] 2024-08-08T21:00:15.9214141Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 74%] 2024-08-08T21:00:15.9215729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 74%] 2024-08-08T21:00:15.9217251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 74%] 2024-08-08T21:00:15.9218775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0064s] [ 74%] 2024-08-08T21:00:15.9220310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 74%] 2024-08-08T21:00:15.9221843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 74%] 2024-08-08T21:00:15.9223359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 74%] 2024-08-08T21:00:15.9224935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 74%] 2024-08-08T21:00:15.9226518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 74%] 2024-08-08T21:00:15.9228038Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 74%] 2024-08-08T21:00:15.9229580Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 74%] 2024-08-08T21:00:15.9231096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 74%] 2024-08-08T21:00:15.9232646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 74%] 2024-08-08T21:00:15.9234182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 74%] 2024-08-08T21:00:15.9235706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 74%] 2024-08-08T21:00:15.9237272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0084s] [ 74%] 2024-08-08T21:00:15.9238872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 74%] 2024-08-08T21:00:15.9240383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 74%] 2024-08-08T21:00:15.9241929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 74%] 2024-08-08T21:00:15.9243439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 74%] 2024-08-08T21:00:15.9244990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 74%] 2024-08-08T21:00:15.9246499Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 74%] 2024-08-08T21:00:15.9248062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 74%] 2024-08-08T21:00:15.9249648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0084s] [ 74%] 2024-08-08T21:00:15.9251177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 74%] 2024-08-08T21:00:15.9252707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 74%] 2024-08-08T21:00:15.9254273Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 74%] 2024-08-08T21:00:15.9255820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 74%] 2024-08-08T21:00:15.9257348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 74%] 2024-08-08T21:00:15.9259223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 74%] 2024-08-08T21:00:15.9260890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 74%] 2024-08-08T21:00:15.9262487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0087s] [ 74%] 2024-08-08T21:00:15.9264034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0085s] [ 74%] 2024-08-08T21:00:15.9265559Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 74%] 2024-08-08T21:00:15.9267106Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 74%] 2024-08-08T21:00:15.9268613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 74%] 2024-08-08T21:00:15.9270153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 74%] 2024-08-08T21:00:15.9271726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 74%] 2024-08-08T21:00:15.9273345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 74%] 2024-08-08T21:00:15.9274887Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0094s] [ 74%] 2024-08-08T21:00:15.9276413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 74%] 2024-08-08T21:00:15.9277935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 74%] 2024-08-08T21:00:15.9279470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 74%] 2024-08-08T21:00:15.9281001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 74%] 2024-08-08T21:00:15.9282526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 74%] 2024-08-08T21:00:15.9284107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 74%] 2024-08-08T21:00:15.9285676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 74%] 2024-08-08T21:00:15.9287213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 74%] 2024-08-08T21:00:15.9288776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 74%] 2024-08-08T21:00:15.9290283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 74%] 2024-08-08T21:00:15.9291828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 74%] 2024-08-08T21:00:15.9293325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 74%] 2024-08-08T21:00:15.9294907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 74%] 2024-08-08T21:00:15.9296468Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 74%] 2024-08-08T21:00:15.9298004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 74%] 2024-08-08T21:00:15.9299510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 74%] 2024-08-08T21:00:15.9301050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 74%] 2024-08-08T21:00:15.9302573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 74%] 2024-08-08T21:00:15.9304116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_128_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 74%] 2024-08-08T21:00:15.9305645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0165s] [ 74%] 2024-08-08T21:00:15.9307239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0169s] [ 74%] 2024-08-08T21:00:15.9308834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0115s] [ 74%] 2024-08-08T21:00:15.9310369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0116s] [ 74%] 2024-08-08T21:00:15.9311920Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0217s] [ 74%] 2024-08-08T21:00:15.9313467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0220s] [ 74%] 2024-08-08T21:00:15.9315027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0141s] [ 74%] 2024-08-08T21:00:15.9316571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0143s] [ 74%] 2024-08-08T21:00:15.9318147Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0164s] [ 74%] 2024-08-08T21:00:15.9319724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0166s] [ 74%] 2024-08-08T21:00:15.9321241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0118s] [ 74%] 2024-08-08T21:00:15.9322781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0119s] [ 74%] 2024-08-08T21:00:15.9324303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0216s] [ 74%] 2024-08-08T21:00:15.9325859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0218s] [ 74%] 2024-08-08T21:00:15.9327381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0144s] [ 74%] 2024-08-08T21:00:15.9328930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0138s] [ 74%] 2024-08-08T21:00:15.9330494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0172s] [ 75%] 2024-08-08T21:00:15.9332092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0180s] [ 75%] 2024-08-08T21:00:15.9333606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0124s] [ 75%] 2024-08-08T21:00:15.9335159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0129s] [ 75%] 2024-08-08T21:00:15.9336715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0230s] [ 75%] 2024-08-08T21:00:15.9338237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0231s] [ 75%] 2024-08-08T21:00:15.9339801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0151s] [ 75%] 2024-08-08T21:00:15.9341394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0155s] [ 75%] 2024-08-08T21:00:15.9342984Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0180s] [ 75%] 2024-08-08T21:00:15.9344545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1912s] [ 75%] 2024-08-08T21:00:15.9346072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0134s] [ 75%] 2024-08-08T21:00:15.9347584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 75%] 2024-08-08T21:00:15.9349132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0227s] [ 75%] 2024-08-08T21:00:15.9350668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0234s] [ 75%] 2024-08-08T21:00:15.9352201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0161s] [ 75%] 2024-08-08T21:00:15.9353769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0162s] [ 75%] 2024-08-08T21:00:15.9355387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0211s] [ 75%] 2024-08-08T21:00:15.9356942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0223s] [ 75%] 2024-08-08T21:00:15.9358783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0159s] [ 75%] 2024-08-08T21:00:15.9360418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0157s] [ 75%] 2024-08-08T21:00:15.9361956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0269s] [ 75%] 2024-08-08T21:00:15.9363513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0271s] [ 75%] 2024-08-08T21:00:15.9365139Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0193s] [ 75%] 2024-08-08T21:00:15.9366804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0194s] [ 75%] 2024-08-08T21:00:15.9368312Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0206s] [ 75%] 2024-08-08T21:00:15.9369841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0219s] [ 75%] 2024-08-08T21:00:15.9371379Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0155s] [ 75%] 2024-08-08T21:00:15.9372910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0159s] [ 75%] 2024-08-08T21:00:15.9374443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0265s] [ 75%] 2024-08-08T21:00:15.9375972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0267s] [ 75%] 2024-08-08T21:00:15.9377586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0200s] [ 75%] 2024-08-08T21:00:15.9379182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0200s] [ 75%] 2024-08-08T21:00:15.9380713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0158s] [ 75%] 2024-08-08T21:00:15.9382236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0160s] [ 75%] 2024-08-08T21:00:15.9383773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0114s] [ 75%] 2024-08-08T21:00:15.9385297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0115s] [ 75%] 2024-08-08T21:00:15.9386816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0211s] [ 75%] 2024-08-08T21:00:15.9388408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0210s] [ 75%] 2024-08-08T21:00:15.9389993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0138s] [ 75%] 2024-08-08T21:00:15.9391546Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1845s] [ 75%] 2024-08-08T21:00:15.9393049Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0159s] [ 75%] 2024-08-08T21:00:15.9394599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0161s] [ 75%] 2024-08-08T21:00:15.9396126Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0117s] [ 75%] 2024-08-08T21:00:15.9397668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0118s] [ 75%] 2024-08-08T21:00:15.9399186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0213s] [ 75%] 2024-08-08T21:00:15.9400757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0215s] [ 75%] 2024-08-08T21:00:15.9402349Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0142s] [ 75%] 2024-08-08T21:00:15.9403882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0141s] [ 75%] 2024-08-08T21:00:15.9405433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0290s] [ 75%] 2024-08-08T21:00:15.9406953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0307s] [ 75%] 2024-08-08T21:00:15.9408495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0186s] [ 75%] 2024-08-08T21:00:15.9410026Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0181s] [ 75%] 2024-08-08T21:00:15.9411614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0391s] [ 75%] 2024-08-08T21:00:15.9413202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0393s] [ 75%] 2024-08-08T21:00:15.9414770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0244s] [ 75%] 2024-08-08T21:00:15.9416317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0243s] [ 75%] 2024-08-08T21:00:15.9417844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0266s] [ 75%] 2024-08-08T21:00:15.9419398Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0262s] [ 75%] 2024-08-08T21:00:15.9420910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0195s] [ 75%] 2024-08-08T21:00:15.9422451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0196s] [ 75%] 2024-08-08T21:00:15.9424019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0348s] [ 75%] 2024-08-08T21:00:15.9425626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0348s] [ 75%] 2024-08-08T21:00:15.9427143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0236s] [ 75%] 2024-08-08T21:00:15.9428692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0230s] [ 75%] 2024-08-08T21:00:15.9430242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0305s] [ 75%] 2024-08-08T21:00:15.9431765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0323s] [ 75%] 2024-08-08T21:00:15.9433281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.1888s] [ 75%] 2024-08-08T21:00:15.9434858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0206s] [ 75%] 2024-08-08T21:00:15.9436459Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0413s] [ 75%] 2024-08-08T21:00:15.9438007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0421s] [ 75%] 2024-08-08T21:00:15.9439571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0265s] [ 75%] 2024-08-08T21:00:15.9441124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0264s] [ 75%] 2024-08-08T21:00:15.9442679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0298s] [ 75%] 2024-08-08T21:00:15.9444203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0297s] [ 75%] 2024-08-08T21:00:15.9445735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0217s] [ 75%] 2024-08-08T21:00:15.9447308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0217s] [ 75%] 2024-08-08T21:00:15.9448878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0362s] [ 75%] 2024-08-08T21:00:15.9450426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0365s] [ 75%] 2024-08-08T21:00:15.9451948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0256s] [ 76%] 2024-08-08T21:00:15.9453500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0256s] [ 76%] 2024-08-08T21:00:15.9455060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0365s] [ 76%] 2024-08-08T21:00:15.9456606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0390s] [ 76%] 2024-08-08T21:00:15.9458166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0252s] [ 76%] 2024-08-08T21:00:15.9460140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0251s] [ 76%] 2024-08-08T21:00:15.9461676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0469s] [ 76%] 2024-08-08T21:00:15.9463234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0472s] [ 76%] 2024-08-08T21:00:15.9464783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.2022s] [ 76%] 2024-08-08T21:00:15.9466335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0339s] [ 76%] 2024-08-08T21:00:15.9467898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0332s] [ 76%] 2024-08-08T21:00:15.9469400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0349s] [ 76%] 2024-08-08T21:00:15.9471047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0251s] [ 76%] 2024-08-08T21:00:15.9473328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0250s] [ 76%] 2024-08-08T21:00:15.9474880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0422s] [ 76%] 2024-08-08T21:00:15.9476406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0414s] [ 76%] 2024-08-08T21:00:15.9477954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0306s] [ 76%] 2024-08-08T21:00:15.9479486Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0314s] [ 76%] 2024-08-08T21:00:15.9481023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0272s] [ 76%] 2024-08-08T21:00:15.9482633Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0280s] [ 76%] 2024-08-08T21:00:15.9484219Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0175s] [ 76%] 2024-08-08T21:00:15.9485757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0178s] [ 76%] 2024-08-08T21:00:15.9487274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0381s] [ 76%] 2024-08-08T21:00:15.9488826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0385s] [ 76%] 2024-08-08T21:00:15.9490354Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0227s] [ 76%] 2024-08-08T21:00:15.9491900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0227s] [ 76%] 2024-08-08T21:00:15.9493409Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0242s] [ 76%] 2024-08-08T21:00:15.9495005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0249s] [ 76%] 2024-08-08T21:00:15.9496571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.1830s] [ 76%] 2024-08-08T21:00:15.9498099Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0179s] [ 76%] 2024-08-08T21:00:15.9499645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0322s] [ 76%] 2024-08-08T21:00:15.9501179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0325s] [ 76%] 2024-08-08T21:00:15.9502718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0221s] [ 76%] 2024-08-08T21:00:15.9504239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0223s] [ 76%] 2024-08-08T21:00:15.9505820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0086s] [ 76%] 2024-08-08T21:00:15.9507408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 76%] 2024-08-08T21:00:15.9508953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 76%] 2024-08-08T21:00:15.9510469Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 76%] 2024-08-08T21:00:15.9512019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0109s] [ 76%] 2024-08-08T21:00:15.9513558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 76%] 2024-08-08T21:00:15.9515075Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0077s] [ 76%] 2024-08-08T21:00:15.9516622Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 76%] 2024-08-08T21:00:15.9518184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0085s] [ 76%] 2024-08-08T21:00:15.9519776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 76%] 2024-08-08T21:00:15.9521289Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 76%] 2024-08-08T21:00:15.9522834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 76%] 2024-08-08T21:00:15.9524360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0108s] [ 76%] 2024-08-08T21:00:15.9525912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 76%] 2024-08-08T21:00:15.9527432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0086s] [ 76%] 2024-08-08T21:00:15.9529003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 76%] 2024-08-08T21:00:15.9530588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0093s] [ 76%] 2024-08-08T21:00:15.9532130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0093s] [ 76%] 2024-08-08T21:00:15.9533686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0076s] [ 76%] 2024-08-08T21:00:15.9535226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 76%] 2024-08-08T21:00:15.9536753Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0118s] [ 76%] 2024-08-08T21:00:15.9538280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0116s] [ 76%] 2024-08-08T21:00:15.9539813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 76%] 2024-08-08T21:00:15.9541399Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0097s] [ 76%] 2024-08-08T21:00:15.9542990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0097s] [ 76%] 2024-08-08T21:00:15.9544513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 76%] 2024-08-08T21:00:15.9546031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0069s] [ 76%] 2024-08-08T21:00:15.9547581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 76%] 2024-08-08T21:00:15.9549103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0108s] [ 76%] 2024-08-08T21:00:15.9550654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0115s] [ 76%] 2024-08-08T21:00:15.9552228Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0079s] [ 76%] 2024-08-08T21:00:15.9553834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 76%] 2024-08-08T21:00:15.9555400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0111s] [ 76%] 2024-08-08T21:00:15.9556950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 76%] 2024-08-08T21:00:15.9558809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0086s] [ 76%] 2024-08-08T21:00:15.9560389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 76%] 2024-08-08T21:00:15.9561929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0128s] [ 76%] 2024-08-08T21:00:15.9563455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0129s] [ 76%] 2024-08-08T21:00:15.9565080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 76%] 2024-08-08T21:00:15.9566750Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0089s] [ 76%] 2024-08-08T21:00:15.9568278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0108s] [ 76%] 2024-08-08T21:00:15.9569811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0109s] [ 76%] 2024-08-08T21:00:15.9571348Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0086s] [ 76%] 2024-08-08T21:00:15.9572873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 77%] 2024-08-08T21:00:15.9574403Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0131s] [ 77%] 2024-08-08T21:00:15.9575998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0131s] [ 77%] 2024-08-08T21:00:15.9577590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0097s] [ 77%] 2024-08-08T21:00:15.9579138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0096s] [ 77%] 2024-08-08T21:00:15.9580651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 77%] 2024-08-08T21:00:15.9582204Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 77%] 2024-08-08T21:00:15.9583723Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 77%] 2024-08-08T21:00:15.9585271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 77%] 2024-08-08T21:00:15.9586784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0099s] [ 77%] 2024-08-08T21:00:15.9588386Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0099s] [ 77%] 2024-08-08T21:00:15.9589964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 77%] 2024-08-08T21:00:15.9591510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 77%] 2024-08-08T21:00:15.9593033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0080s] [ 77%] 2024-08-08T21:00:15.9594538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 77%] 2024-08-08T21:00:15.9596062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 77%] 2024-08-08T21:00:15.9597578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 77%] 2024-08-08T21:00:15.9599155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0103s] [ 77%] 2024-08-08T21:00:15.9600729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 77%] 2024-08-08T21:00:15.9602264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 77%] 2024-08-08T21:00:15.9603785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 77%] 2024-08-08T21:00:15.9605339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0527s] [ 77%] 2024-08-08T21:00:15.9606879Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0559s] [ 77%] 2024-08-08T21:00:15.9608406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0320s] [ 77%] 2024-08-08T21:00:15.9609952Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0321s] [ 77%] 2024-08-08T21:00:15.9611535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0763s] [ 77%] 2024-08-08T21:00:15.9613143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.2397s] [ 77%] 2024-08-08T21:00:15.9614681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0439s] [ 77%] 2024-08-08T21:00:15.9616244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0432s] [ 77%] 2024-08-08T21:00:15.9617777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0401s] [ 77%] 2024-08-08T21:00:15.9619328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0402s] [ 77%] 2024-08-08T21:00:15.9620840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0324s] [ 77%] 2024-08-08T21:00:15.9622419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0325s] [ 77%] 2024-08-08T21:00:15.9624016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0514s] [ 77%] 2024-08-08T21:00:15.9625552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0519s] [ 77%] 2024-08-08T21:00:15.9627090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0399s] [ 77%] 2024-08-08T21:00:15.9628720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1965s] [ 77%] 2024-08-08T21:00:15.9630167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0584s] [ 77%] 2024-08-08T21:00:15.9631701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0618s] [ 77%] 2024-08-08T21:00:15.9633235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0369s] [ 77%] 2024-08-08T21:00:15.9634813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0366s] [ 77%] 2024-08-08T21:00:15.9636413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0778s] [ 77%] 2024-08-08T21:00:15.9637948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0795s] [ 77%] 2024-08-08T21:00:15.9639491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0469s] [ 77%] 2024-08-08T21:00:15.9641071Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0474s] [ 77%] 2024-08-08T21:00:15.9642590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.2006s] [ 77%] 2024-08-08T21:00:15.9644134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0453s] [ 77%] 2024-08-08T21:00:15.9645714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0363s] [ 77%] 2024-08-08T21:00:15.9647301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0365s] [ 77%] 2024-08-08T21:00:15.9648826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0556s] [ 77%] 2024-08-08T21:00:15.9650369Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0556s] [ 77%] 2024-08-08T21:00:15.9651897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0431s] [ 77%] 2024-08-08T21:00:15.9653451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0437s] [ 77%] 2024-08-08T21:00:15.9655017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0684s] [ 77%] 2024-08-08T21:00:15.9656555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.2247s] [ 77%] 2024-08-08T21:00:15.9658157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0430s] [ 77%] 2024-08-08T21:00:15.9660129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0443s] [ 77%] 2024-08-08T21:00:15.9661697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0889s] [ 77%] 2024-08-08T21:00:15.9663239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0902s] [ 77%] 2024-08-08T21:00:15.9664809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0627s] [ 77%] 2024-08-08T21:00:15.9666359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0635s] [ 77%] 2024-08-08T21:00:15.9667895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0519s] [ 77%] 2024-08-08T21:00:15.9669509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.2135s] [ 77%] 2024-08-08T21:00:15.9671102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0435s] [ 77%] 2024-08-08T21:00:15.9672646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0434s] [ 77%] 2024-08-08T21:00:15.9674191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0633s] [ 77%] 2024-08-08T21:00:15.9675724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0634s] [ 77%] 2024-08-08T21:00:15.9677249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0539s] [ 77%] 2024-08-08T21:00:15.9678798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0534s] [ 77%] 2024-08-08T21:00:15.9680314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0507s] [ 77%] 2024-08-08T21:00:15.9681918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.2107s] [ 77%] 2024-08-08T21:00:15.9683503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0302s] [ 77%] 2024-08-08T21:00:15.9685040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0301s] [ 77%] 2024-08-08T21:00:15.9686562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0721s] [ 77%] 2024-08-08T21:00:15.9688097Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0736s] [ 77%] 2024-08-08T21:00:15.9689640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0427s] [ 77%] 2024-08-08T21:00:15.9691164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0413s] [ 77%] 2024-08-08T21:00:15.9692731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0378s] [ 77%] 2024-08-08T21:00:15.9694309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0389s] [ 78%] 2024-08-08T21:00:15.9695842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0321s] [ 78%] 2024-08-08T21:00:15.9697365Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0317s] [ 78%] 2024-08-08T21:00:15.9698907Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0481s] [ 78%] 2024-08-08T21:00:15.9700440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.2137s] [ 78%] 2024-08-08T21:00:15.9701977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0387s] [ 78%] 2024-08-08T21:00:15.9703496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0387s] [ 78%] 2024-08-08T21:00:15.9705052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0110s] [ 78%] 2024-08-08T21:00:15.9706646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0110s] [ 78%] 2024-08-08T21:00:15.9708161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0087s] [ 78%] 2024-08-08T21:00:15.9709703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 78%] 2024-08-08T21:00:15.9711350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0131s] [ 78%] 2024-08-08T21:00:15.9712784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0131s] [ 78%] 2024-08-08T21:00:15.9714306Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0098s] [ 78%] 2024-08-08T21:00:15.9715892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0098s] [ 78%] 2024-08-08T21:00:15.9717452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0108s] [ 78%] 2024-08-08T21:00:15.9718986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 78%] 2024-08-08T21:00:15.9720515Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0087s] [ 78%] 2024-08-08T21:00:15.9722047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0089s] [ 78%] 2024-08-08T21:00:15.9723594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0138s] [ 78%] 2024-08-08T21:00:15.9725121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0141s] [ 78%] 2024-08-08T21:00:15.9726655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0106s] [ 78%] 2024-08-08T21:00:15.9728248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0105s] [ 78%] 2024-08-08T21:00:15.9729799Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0118s] [ 78%] 2024-08-08T21:00:15.9731321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0116s] [ 78%] 2024-08-08T21:00:15.9732854Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0088s] [ 78%] 2024-08-08T21:00:15.9734383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 78%] 2024-08-08T21:00:15.9735904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0140s] [ 78%] 2024-08-08T21:00:15.9737450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0144s] [ 78%] 2024-08-08T21:00:15.9739012Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0107s] [ 78%] 2024-08-08T21:00:15.9740626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0108s] [ 78%] 2024-08-08T21:00:15.9742142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0120s] [ 78%] 2024-08-08T21:00:15.9743692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0122s] [ 78%] 2024-08-08T21:00:15.9745226Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0096s] [ 78%] 2024-08-08T21:00:15.9746762Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0097s] [ 78%] 2024-08-08T21:00:15.9748286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0144s] [ 78%] 2024-08-08T21:00:15.9749808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0138s] [ 78%] 2024-08-08T21:00:15.9751410Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0101s] [ 78%] 2024-08-08T21:00:15.9752965Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0100s] [ 78%] 2024-08-08T21:00:15.9754528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0135s] [ 78%] 2024-08-08T21:00:15.9756053Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0137s] [ 78%] 2024-08-08T21:00:15.9757592Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0102s] [ 78%] 2024-08-08T21:00:15.9759500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 78%] 2024-08-08T21:00:15.9761067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0163s] [ 78%] 2024-08-08T21:00:15.9762685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0165s] [ 78%] 2024-08-08T21:00:15.9764301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0127s] [ 78%] 2024-08-08T21:00:15.9765834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0126s] [ 78%] 2024-08-08T21:00:15.9767345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0141s] [ 78%] 2024-08-08T21:00:15.9768902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0140s] [ 78%] 2024-08-08T21:00:15.9770427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0100s] [ 78%] 2024-08-08T21:00:15.9771988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0100s] [ 78%] 2024-08-08T21:00:15.9773494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0159s] [ 78%] 2024-08-08T21:00:15.9775105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0165s] [ 78%] 2024-08-08T21:00:15.9776694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0127s] [ 78%] 2024-08-08T21:00:15.9778234Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1836s] [ 78%] 2024-08-08T21:00:15.9779754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0101s] [ 78%] 2024-08-08T21:00:15.9781300Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 78%] 2024-08-08T21:00:15.9782839Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 78%] 2024-08-08T21:00:15.9784366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 78%] 2024-08-08T21:00:15.9785951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0129s] [ 78%] 2024-08-08T21:00:15.9787533Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0129s] [ 78%] 2024-08-08T21:00:15.9789076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0095s] [ 78%] 2024-08-08T21:00:15.9790604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0097s] [ 78%] 2024-08-08T21:00:15.9792133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0105s] [ 78%] 2024-08-08T21:00:15.9793671Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 78%] 2024-08-08T21:00:15.9795198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 78%] 2024-08-08T21:00:15.9796715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0083s] [ 78%] 2024-08-08T21:00:15.9798281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0132s] [ 78%] 2024-08-08T21:00:15.9799894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0130s] [ 78%] 2024-08-08T21:00:15.9801419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0096s] [ 78%] 2024-08-08T21:00:15.9802968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0098s] [ 78%] 2024-08-08T21:00:15.9804500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0085s] [ 78%] 2024-08-08T21:00:15.9806060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 78%] 2024-08-08T21:00:15.9807575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0079s] [ 78%] 2024-08-08T21:00:15.9809167Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 78%] 2024-08-08T21:00:15.9810742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0111s] [ 78%] 2024-08-08T21:00:15.9812280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0110s] [ 78%] 2024-08-08T21:00:15.9813816Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 78%] 2024-08-08T21:00:15.9815350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 79%] 2024-08-08T21:00:15.9816880Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0091s] [ 79%] 2024-08-08T21:00:15.9818405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 79%] 2024-08-08T21:00:15.9819939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0077s] [ 79%] 2024-08-08T21:00:15.9821537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 79%] 2024-08-08T21:00:15.9823101Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0116s] [ 79%] 2024-08-08T21:00:15.9824630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0112s] [ 79%] 2024-08-08T21:00:15.9826168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 79%] 2024-08-08T21:00:15.9827703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 79%] 2024-08-08T21:00:15.9829221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0087s] [ 79%] 2024-08-08T21:00:15.9830770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0087s] [ 79%] 2024-08-08T21:00:15.9832346Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 79%] 2024-08-08T21:00:15.9833947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 79%] 2024-08-08T21:00:15.9835475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0112s] [ 79%] 2024-08-08T21:00:15.9837024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0113s] [ 79%] 2024-08-08T21:00:15.9838548Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0082s] [ 79%] 2024-08-08T21:00:15.9840102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0079s] [ 79%] 2024-08-08T21:00:15.9841608Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0092s] [ 79%] 2024-08-08T21:00:15.9843154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 79%] 2024-08-08T21:00:15.9844716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 79%] 2024-08-08T21:00:15.9846293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 79%] 2024-08-08T21:00:15.9847825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0117s] [ 79%] 2024-08-08T21:00:15.9849357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 79%] 2024-08-08T21:00:15.9850898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0084s] [ 79%] 2024-08-08T21:00:15.9852438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0083s] [ 79%] 2024-08-08T21:00:15.9853971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0109s] [ 79%] 2024-08-08T21:00:15.9855596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0110s] [ 79%] 2024-08-08T21:00:15.9857190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 79%] 2024-08-08T21:00:15.9859062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0080s] [ 79%] 2024-08-08T21:00:15.9860614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0127s] [ 79%] 2024-08-08T21:00:15.9862163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0129s] [ 79%] 2024-08-08T21:00:15.9863692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0091s] [ 79%] 2024-08-08T21:00:15.9865242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 79%] 2024-08-08T21:00:15.9866742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0114s] [ 79%] 2024-08-08T21:00:15.9868371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0114s] [ 79%] 2024-08-08T21:00:15.9869956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0085s] [ 79%] 2024-08-08T21:00:15.9871493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 79%] 2024-08-08T21:00:15.9873003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0133s] [ 79%] 2024-08-08T21:00:15.9874558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0133s] [ 79%] 2024-08-08T21:00:15.9876083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0099s] [ 79%] 2024-08-08T21:00:15.9877610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0091s] [ 79%] 2024-08-08T21:00:15.9879190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0077s] [ 79%] 2024-08-08T21:00:15.9880790Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 79%] 2024-08-08T21:00:15.9882322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 79%] 2024-08-08T21:00:15.9883837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 79%] 2024-08-08T21:00:15.9885366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0099s] [ 79%] 2024-08-08T21:00:15.9886898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0098s] [ 79%] 2024-08-08T21:00:15.9888430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 79%] 2024-08-08T21:00:15.9889953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 79%] 2024-08-08T21:00:15.9891498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0081s] [ 79%] 2024-08-08T21:00:15.9893087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 79%] 2024-08-08T21:00:15.9894596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 79%] 2024-08-08T21:00:15.9896129Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 79%] 2024-08-08T21:00:15.9897647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0101s] [ 79%] 2024-08-08T21:00:15.9899200Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 79%] 2024-08-08T21:00:15.9900730Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 79%] 2024-08-08T21:00:15.9902291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_256_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 79%] 2024-08-08T21:00:15.9903856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 79%] 2024-08-08T21:00:15.9905411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 79%] 2024-08-08T21:00:15.9906922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 79%] 2024-08-08T21:00:15.9908470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 79%] 2024-08-08T21:00:15.9909991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 79%] 2024-08-08T21:00:15.9911523Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 79%] 2024-08-08T21:00:15.9913073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 79%] 2024-08-08T21:00:15.9914579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 79%] 2024-08-08T21:00:15.9916143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 79%] 2024-08-08T21:00:15.9917726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 79%] 2024-08-08T21:00:15.9919253Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 79%] 2024-08-08T21:00:15.9920773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 79%] 2024-08-08T21:00:15.9922294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 79%] 2024-08-08T21:00:15.9923843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 79%] 2024-08-08T21:00:15.9925350Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 79%] 2024-08-08T21:00:15.9926942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 79%] 2024-08-08T21:00:15.9928510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 79%] 2024-08-08T21:00:15.9930058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 79%] 2024-08-08T21:00:15.9931570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 79%] 2024-08-08T21:00:15.9933119Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 79%] 2024-08-08T21:00:15.9934660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0070s] [ 79%] 2024-08-08T21:00:15.9936183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 80%] 2024-08-08T21:00:15.9937695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 80%] 2024-08-08T21:00:15.9939320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 80%] 2024-08-08T21:00:15.9940888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 80%] 2024-08-08T21:00:15.9942411Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 80%] 2024-08-08T21:00:15.9943969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 80%] 2024-08-08T21:00:15.9945471Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 80%] 2024-08-08T21:00:15.9947005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 80%] 2024-08-08T21:00:15.9948527Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 80%] 2024-08-08T21:00:15.9950105Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 80%] 2024-08-08T21:00:15.9951684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 80%] 2024-08-08T21:00:15.9953198Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 80%] 2024-08-08T21:00:15.9954789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 80%] 2024-08-08T21:00:15.9956317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 80%] 2024-08-08T21:00:15.9957864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 80%] 2024-08-08T21:00:15.9959764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 80%] 2024-08-08T21:00:15.9961343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 80%] 2024-08-08T21:00:15.9962948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 80%] 2024-08-08T21:00:15.9964577Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 80%] 2024-08-08T21:00:15.9966087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0066s] [ 80%] 2024-08-08T21:00:15.9967635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 80%] 2024-08-08T21:00:15.9969153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 80%] 2024-08-08T21:00:15.9970676Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 80%] 2024-08-08T21:00:15.9972207Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 80%] 2024-08-08T21:00:15.9973795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 80%] 2024-08-08T21:00:15.9975401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 80%] 2024-08-08T21:00:15.9976930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 80%] 2024-08-08T21:00:15.9978461Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 80%] 2024-08-08T21:00:15.9980002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 80%] 2024-08-08T21:00:15.9981538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 80%] 2024-08-08T21:00:15.9983056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 80%] 2024-08-08T21:00:15.9984576Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 80%] 2024-08-08T21:00:15.9986171Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 80%] 2024-08-08T21:00:15.9987763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 80%] 2024-08-08T21:00:15.9989302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 80%] 2024-08-08T21:00:15.9990795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 80%] 2024-08-08T21:00:15.9992341Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 80%] 2024-08-08T21:00:15.9993852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 80%] 2024-08-08T21:00:15.9995383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 80%] 2024-08-08T21:00:15.9996942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 80%] 2024-08-08T21:00:15.9998534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 80%] 2024-08-08T21:00:16.0000045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 80%] 2024-08-08T21:00:16.0001570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 80%] 2024-08-08T21:00:16.0003109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0070s] [ 80%] 2024-08-08T21:00:16.0004655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 80%] 2024-08-08T21:00:16.0006190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 80%] 2024-08-08T21:00:16.0007721Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 80%] 2024-08-08T21:00:16.0009331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 80%] 2024-08-08T21:00:16.0010905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 80%] 2024-08-08T21:00:16.0012433Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 80%] 2024-08-08T21:00:16.0013966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 80%] 2024-08-08T21:00:16.0015495Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 80%] 2024-08-08T21:00:16.0017050Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 80%] 2024-08-08T21:00:16.0018558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 80%] 2024-08-08T21:00:16.0020123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 80%] 2024-08-08T21:00:16.0021689Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 80%] 2024-08-08T21:00:16.0023233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 80%] 2024-08-08T21:00:16.0024747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 80%] 2024-08-08T21:00:16.0026298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 80%] 2024-08-08T21:00:16.0027819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0074s] [ 80%] 2024-08-08T21:00:16.0029366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 80%] 2024-08-08T21:00:16.0030889Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 80%] 2024-08-08T21:00:16.0032473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 80%] 2024-08-08T21:00:16.0034064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0082s] [ 80%] 2024-08-08T21:00:16.0035595Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 80%] 2024-08-08T21:00:16.0037130Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 80%] 2024-08-08T21:00:16.0038669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 80%] 2024-08-08T21:00:16.0040201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0065s] [ 80%] 2024-08-08T21:00:16.0041722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 80%] 2024-08-08T21:00:16.0043298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 80%] 2024-08-08T21:00:16.0044890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 80%] 2024-08-08T21:00:16.0046384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 80%] 2024-08-08T21:00:16.0047929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 80%] 2024-08-08T21:00:16.0049457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 80%] 2024-08-08T21:00:16.0051019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 80%] 2024-08-08T21:00:16.0052538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0091s] [ 80%] 2024-08-08T21:00:16.0054094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 80%] 2024-08-08T21:00:16.0055703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 80%] 2024-08-08T21:00:16.0057302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 81%] 2024-08-08T21:00:16.0059183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0103s] [ 81%] 2024-08-08T21:00:16.0060770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 81%] 2024-08-08T21:00:16.0062296Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0076s] [ 81%] 2024-08-08T21:00:16.0063825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 81%] 2024-08-08T21:00:16.0065343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0076s] [ 81%] 2024-08-08T21:00:16.0066954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 81%] 2024-08-08T21:00:16.0068558Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 81%] 2024-08-08T21:00:16.0070081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 81%] 2024-08-08T21:00:16.0071609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0081s] [ 81%] 2024-08-08T21:00:16.0073140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 81%] 2024-08-08T21:00:16.0074679Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0077s] [ 81%] 2024-08-08T21:00:16.0076201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 81%] 2024-08-08T21:00:16.0077709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0067s] [ 81%] 2024-08-08T21:00:16.0079322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 81%] 2024-08-08T21:00:16.0080899Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 81%] 2024-08-08T21:00:16.0082439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 81%] 2024-08-08T21:00:16.0083953Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 81%] 2024-08-08T21:00:16.0085509Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 81%] 2024-08-08T21:00:16.0087025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 81%] 2024-08-08T21:00:16.0088566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 81%] 2024-08-08T21:00:16.0090113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 81%] 2024-08-08T21:00:16.0091715Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 81%] 2024-08-08T21:00:16.0093224Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 81%] 2024-08-08T21:00:16.0094745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 81%] 2024-08-08T21:00:16.0096274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 81%] 2024-08-08T21:00:16.0097803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 81%] 2024-08-08T21:00:16.0099326Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 81%] 2024-08-08T21:00:16.0100843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 81%] 2024-08-08T21:00:16.0102417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 81%] 2024-08-08T21:00:16.0104001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 81%] 2024-08-08T21:00:16.0105513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 81%] 2024-08-08T21:00:16.0107034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 81%] 2024-08-08T21:00:16.0108556Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 81%] 2024-08-08T21:00:16.0110100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 81%] 2024-08-08T21:00:16.0111605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 81%] 2024-08-08T21:00:16.0113187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 81%] 2024-08-08T21:00:16.0114740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 81%] 2024-08-08T21:00:16.0116271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 81%] 2024-08-08T21:00:16.0117768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 81%] 2024-08-08T21:00:16.0119291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 81%] 2024-08-08T21:00:16.0120804Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 81%] 2024-08-08T21:00:16.0122339Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 81%] 2024-08-08T21:00:16.0123841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 81%] 2024-08-08T21:00:16.0125400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 81%] 2024-08-08T21:00:16.0126998Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 81%] 2024-08-08T21:00:16.0128494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 81%] 2024-08-08T21:00:16.0130015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 81%] 2024-08-08T21:00:16.0131531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 81%] 2024-08-08T21:00:16.0133072Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 81%] 2024-08-08T21:00:16.0134597Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 81%] 2024-08-08T21:00:16.0136164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 81%] 2024-08-08T21:00:16.0137757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 81%] 2024-08-08T21:00:16.0139236Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 81%] 2024-08-08T21:00:16.0140759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 81%] 2024-08-08T21:00:16.0142259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 81%] 2024-08-08T21:00:16.0143794Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 81%] 2024-08-08T21:00:16.0145302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 81%] 2024-08-08T21:00:16.0146830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 81%] 2024-08-08T21:00:16.0148372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 81%] 2024-08-08T21:00:16.0149955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 81%] 2024-08-08T21:00:16.0151458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 81%] 2024-08-08T21:00:16.0152993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 81%] 2024-08-08T21:00:16.0154540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 81%] 2024-08-08T21:00:16.0156067Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 81%] 2024-08-08T21:00:16.0157603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 81%] 2024-08-08T21:00:16.0159578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 81%] 2024-08-08T21:00:16.0161194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 81%] 2024-08-08T21:00:16.0162724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 81%] 2024-08-08T21:00:16.0164237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 81%] 2024-08-08T21:00:16.0165749Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 81%] 2024-08-08T21:00:16.0167275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 81%] 2024-08-08T21:00:16.0168787Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 81%] 2024-08-08T21:00:16.0170302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 81%] 2024-08-08T21:00:16.0171850Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 81%] 2024-08-08T21:00:16.0173420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 81%] 2024-08-08T21:00:16.0175003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 81%] 2024-08-08T21:00:16.0176503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 81%] 2024-08-08T21:00:16.0178039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 82%] 2024-08-08T21:00:16.0179547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 82%] 2024-08-08T21:00:16.0181093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 82%] 2024-08-08T21:00:16.0182575Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 82%] 2024-08-08T21:00:16.0184157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 82%] 2024-08-08T21:00:16.0185737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 82%] 2024-08-08T21:00:16.0187252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 82%] 2024-08-08T21:00:16.0188771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 82%] 2024-08-08T21:00:16.0190277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 82%] 2024-08-08T21:00:16.0191802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 82%] 2024-08-08T21:00:16.0193316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 82%] 2024-08-08T21:00:16.0194822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 82%] 2024-08-08T21:00:16.0196389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 82%] 2024-08-08T21:00:16.0197951Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 82%] 2024-08-08T21:00:16.0199456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 82%] 2024-08-08T21:00:16.0200977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0105s] [ 82%] 2024-08-08T21:00:16.0202541Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0101s] [ 82%] 2024-08-08T21:00:16.0204082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 82%] 2024-08-08T21:00:16.0205604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 82%] 2024-08-08T21:00:16.0207168Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0110s] [ 82%] 2024-08-08T21:00:16.0208773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0111s] [ 82%] 2024-08-08T21:00:16.0210286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 82%] 2024-08-08T21:00:16.0211822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 82%] 2024-08-08T21:00:16.0213448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0064s] [ 82%] 2024-08-08T21:00:16.0214873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 82%] 2024-08-08T21:00:16.0216400Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 82%] 2024-08-08T21:00:16.0217926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 82%] 2024-08-08T21:00:16.0219512Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 82%] 2024-08-08T21:00:16.0221088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 82%] 2024-08-08T21:00:16.0222615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 82%] 2024-08-08T21:00:16.0224144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 82%] 2024-08-08T21:00:16.0225686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0120s] [ 82%] 2024-08-08T21:00:16.0227222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 82%] 2024-08-08T21:00:16.0228756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 82%] 2024-08-08T21:00:16.0230329Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 82%] 2024-08-08T21:00:16.0231908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0127s] [ 82%] 2024-08-08T21:00:16.0233460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0126s] [ 82%] 2024-08-08T21:00:16.0234973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0080s] [ 82%] 2024-08-08T21:00:16.0236519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 82%] 2024-08-08T21:00:16.0238034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 82%] 2024-08-08T21:00:16.0239573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0073s] [ 82%] 2024-08-08T21:00:16.0241086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 82%] 2024-08-08T21:00:16.0242700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 82%] 2024-08-08T21:00:16.0244237Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0083s] [ 82%] 2024-08-08T21:00:16.0245784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 82%] 2024-08-08T21:00:16.0247309Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 82%] 2024-08-08T21:00:16.0248843Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 82%] 2024-08-08T21:00:16.0250382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0150s] [ 82%] 2024-08-08T21:00:16.0251918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0150s] [ 82%] 2024-08-08T21:00:16.0253491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0091s] [ 82%] 2024-08-08T21:00:16.0255121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 82%] 2024-08-08T21:00:16.0256669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0162s] [ 82%] 2024-08-08T21:00:16.0258195Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0163s] [ 82%] 2024-08-08T21:00:16.0260127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0104s] [ 82%] 2024-08-08T21:00:16.0261675Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0102s] [ 82%] 2024-08-08T21:00:16.0263177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0094s] [ 82%] 2024-08-08T21:00:16.0264722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 82%] 2024-08-08T21:00:16.0266324Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0083s] [ 82%] 2024-08-08T21:00:16.0267956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 82%] 2024-08-08T21:00:16.0269470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0098s] [ 82%] 2024-08-08T21:00:16.0271022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0098s] [ 82%] 2024-08-08T21:00:16.0272555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 82%] 2024-08-08T21:00:16.0274090Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0097s] [ 82%] 2024-08-08T21:00:16.0275624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 82%] 2024-08-08T21:00:16.0277188Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 82%] 2024-08-08T21:00:16.0278792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 82%] 2024-08-08T21:00:16.0280318Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 82%] 2024-08-08T21:00:16.0281848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0103s] [ 82%] 2024-08-08T21:00:16.0283385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0103s] [ 82%] 2024-08-08T21:00:16.0284926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 82%] 2024-08-08T21:00:16.0286448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 82%] 2024-08-08T21:00:16.0287960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 82%] 2024-08-08T21:00:16.0289528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 82%] 2024-08-08T21:00:16.0291103Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 82%] 2024-08-08T21:00:16.0292619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 82%] 2024-08-08T21:00:16.0294134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 82%] 2024-08-08T21:00:16.0295681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 82%] 2024-08-08T21:00:16.0297192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 83%] 2024-08-08T21:00:16.0298725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 83%] 2024-08-08T21:00:16.0300278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 83%] 2024-08-08T21:00:16.0301925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 83%] 2024-08-08T21:00:16.0303441Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 83%] 2024-08-08T21:00:16.0304975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 83%] 2024-08-08T21:00:16.0306490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 83%] 2024-08-08T21:00:16.0308029Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 83%] 2024-08-08T21:00:16.0309564Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 83%] 2024-08-08T21:00:16.0311088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 83%] 2024-08-08T21:00:16.0312654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 83%] 2024-08-08T21:00:16.0314229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 83%] 2024-08-08T21:00:16.0315747Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 83%] 2024-08-08T21:00:16.0317270Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 83%] 2024-08-08T21:00:16.0318809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 83%] 2024-08-08T21:00:16.0320335Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 83%] 2024-08-08T21:00:16.0321861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 83%] 2024-08-08T21:00:16.0323422Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 83%] 2024-08-08T21:00:16.0324979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 83%] 2024-08-08T21:00:16.0326529Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 83%] 2024-08-08T21:00:16.0328059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 83%] 2024-08-08T21:00:16.0329581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 83%] 2024-08-08T21:00:16.0331109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 83%] 2024-08-08T21:00:16.0332686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 83%] 2024-08-08T21:00:16.0334175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 83%] 2024-08-08T21:00:16.0335767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 83%] 2024-08-08T21:00:16.0337319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 83%] 2024-08-08T21:00:16.0338837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 83%] 2024-08-08T21:00:16.0340355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 83%] 2024-08-08T21:00:16.0341877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 83%] 2024-08-08T21:00:16.0343417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 83%] 2024-08-08T21:00:16.0344939Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 83%] 2024-08-08T21:00:16.0346514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 83%] 2024-08-08T21:00:16.0348085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 83%] 2024-08-08T21:00:16.0349613Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 83%] 2024-08-08T21:00:16.0351146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 83%] 2024-08-08T21:00:16.0352677Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 83%] 2024-08-08T21:00:16.0354205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 83%] 2024-08-08T21:00:16.0355764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 83%] 2024-08-08T21:00:16.0357337Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 83%] 2024-08-08T21:00:16.0359206Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 83%] 2024-08-08T21:00:16.0360871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 83%] 2024-08-08T21:00:16.0362454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 83%] 2024-08-08T21:00:16.0363992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 83%] 2024-08-08T21:00:16.0365500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 83%] 2024-08-08T21:00:16.0367043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 83%] 2024-08-08T21:00:16.0368557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 83%] 2024-08-08T21:00:16.0370084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 83%] 2024-08-08T21:00:16.0371701Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 83%] 2024-08-08T21:00:16.0373272Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 83%] 2024-08-08T21:00:16.0374796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 83%] 2024-08-08T21:00:16.0376315Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 83%] 2024-08-08T21:00:16.0377848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 83%] 2024-08-08T21:00:16.0379364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 83%] 2024-08-08T21:00:16.0380895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 83%] 2024-08-08T21:00:16.0382446Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 83%] 2024-08-08T21:00:16.0384011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 83%] 2024-08-08T21:00:16.0385578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 83%] 2024-08-08T21:00:16.0387073Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 83%] 2024-08-08T21:00:16.0388612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 83%] 2024-08-08T21:00:16.0390123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 83%] 2024-08-08T21:00:16.0391654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 83%] 2024-08-08T21:00:16.0393152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 83%] 2024-08-08T21:00:16.0394731Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 83%] 2024-08-08T21:00:16.0396284Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 83%] 2024-08-08T21:00:16.0397809Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 83%] 2024-08-08T21:00:16.0399321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 83%] 2024-08-08T21:00:16.0400847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 83%] 2024-08-08T21:00:16.0402376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 83%] 2024-08-08T21:00:16.0403890Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 83%] 2024-08-08T21:00:16.0405421Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 83%] 2024-08-08T21:00:16.0406991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 83%] 2024-08-08T21:00:16.0408573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 83%] 2024-08-08T21:00:16.0410093Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 83%] 2024-08-08T21:00:16.0411604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 83%] 2024-08-08T21:00:16.0413121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 83%] 2024-08-08T21:00:16.0414645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 83%] 2024-08-08T21:00:16.0416154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 83%] 2024-08-08T21:00:16.0417706Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 84%] 2024-08-08T21:00:16.0419314Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 84%] 2024-08-08T21:00:16.0420793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 84%] 2024-08-08T21:00:16.0422323Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 84%] 2024-08-08T21:00:16.0423837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 84%] 2024-08-08T21:00:16.0425373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 84%] 2024-08-08T21:00:16.0426888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 84%] 2024-08-08T21:00:16.0428426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 84%] 2024-08-08T21:00:16.0429989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 84%] 2024-08-08T21:00:16.0431557Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 84%] 2024-08-08T21:00:16.0433088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 84%] 2024-08-08T21:00:16.0434612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 84%] 2024-08-08T21:00:16.0436133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 84%] 2024-08-08T21:00:16.0437644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 84%] 2024-08-08T21:00:16.0439156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 84%] 2024-08-08T21:00:16.0440710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 84%] 2024-08-08T21:00:16.0442298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 84%] 2024-08-08T21:00:16.0443811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 84%] 2024-08-08T21:00:16.0445307Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 84%] 2024-08-08T21:00:16.0446846Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 84%] 2024-08-08T21:00:16.0448352Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 84%] 2024-08-08T21:00:16.0449884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 84%] 2024-08-08T21:00:16.0451383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 84%] 2024-08-08T21:00:16.0452961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 84%] 2024-08-08T21:00:16.0454531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 84%] 2024-08-08T21:00:16.0456122Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 84%] 2024-08-08T21:00:16.0457640Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 84%] 2024-08-08T21:00:16.0459550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 84%] 2024-08-08T21:00:16.0461079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 84%] 2024-08-08T21:00:16.0462590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 84%] 2024-08-08T21:00:16.0464196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 84%] 2024-08-08T21:00:16.0465784Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 84%] 2024-08-08T21:00:16.0467310Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 84%] 2024-08-08T21:00:16.0468821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 84%] 2024-08-08T21:00:16.0470345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 84%] 2024-08-08T21:00:16.0471865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 84%] 2024-08-08T21:00:16.0473381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 84%] 2024-08-08T21:00:16.0474888Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 84%] 2024-08-08T21:00:16.0476452Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 84%] 2024-08-08T21:00:16.0478057Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 84%] 2024-08-08T21:00:16.0479562Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 84%] 2024-08-08T21:00:16.0481096Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 84%] 2024-08-08T21:00:16.0482601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 84%] 2024-08-08T21:00:16.0484169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 84%] 2024-08-08T21:00:16.0485637Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 84%] 2024-08-08T21:00:16.0487193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 84%] 2024-08-08T21:00:16.0488741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 84%] 2024-08-08T21:00:16.0490269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 84%] 2024-08-08T21:00:16.0491764Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 84%] 2024-08-08T21:00:16.0493276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 84%] 2024-08-08T21:00:16.0494819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 84%] 2024-08-08T21:00:16.0496357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_4_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 84%] 2024-08-08T21:00:16.0497856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0281s] [ 84%] 2024-08-08T21:00:16.0499394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0287s] [ 84%] 2024-08-08T21:00:16.0500985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0178s] [ 84%] 2024-08-08T21:00:16.0502566Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0180s] [ 84%] 2024-08-08T21:00:16.0504110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0394s] [ 84%] 2024-08-08T21:00:16.0505660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0397s] [ 84%] 2024-08-08T21:00:16.0507203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0239s] [ 84%] 2024-08-08T21:00:16.0508768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0241s] [ 84%] 2024-08-08T21:00:16.0510359Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0281s] [ 84%] 2024-08-08T21:00:16.0511928Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0294s] [ 84%] 2024-08-08T21:00:16.0513448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0187s] [ 84%] 2024-08-08T21:00:16.0514986Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0185s] [ 84%] 2024-08-08T21:00:16.0516510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.2068s] [ 84%] 2024-08-08T21:00:16.0518065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0396s] [ 84%] 2024-08-08T21:00:16.0519589Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0251s] [ 84%] 2024-08-08T21:00:16.0521131Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0248s] [ 84%] 2024-08-08T21:00:16.0522720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0317s] [ 84%] 2024-08-08T21:00:16.0524274Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0330s] [ 84%] 2024-08-08T21:00:16.0525807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0213s] [ 84%] 2024-08-08T21:00:16.0527330Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0216s] [ 84%] 2024-08-08T21:00:16.0528878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0435s] [ 84%] 2024-08-08T21:00:16.0530429Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0433s] [ 84%] 2024-08-08T21:00:16.0531989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0273s] [ 84%] 2024-08-08T21:00:16.0533588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0266s] [ 84%] 2024-08-08T21:00:16.0535178Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0311s] [ 84%] 2024-08-08T21:00:16.0536707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0321s] [ 84%] 2024-08-08T21:00:16.0538239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0218s] [ 85%] 2024-08-08T21:00:16.0539765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0219s] [ 85%] 2024-08-08T21:00:16.0541288Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0426s] [ 85%] 2024-08-08T21:00:16.0542842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0432s] [ 85%] 2024-08-08T21:00:16.0544364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0274s] [ 85%] 2024-08-08T21:00:16.0545955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0276s] [ 85%] 2024-08-08T21:00:16.0547538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.2010s] [ 85%] 2024-08-08T21:00:16.0549088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0389s] [ 85%] 2024-08-08T21:00:16.0550599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0255s] [ 85%] 2024-08-08T21:00:16.0552177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0256s] [ 85%] 2024-08-08T21:00:16.0553725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0483s] [ 85%] 2024-08-08T21:00:16.0555303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0483s] [ 85%] 2024-08-08T21:00:16.0556930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0352s] [ 85%] 2024-08-08T21:00:16.0558829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0349s] [ 85%] 2024-08-08T21:00:16.0560474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0370s] [ 85%] 2024-08-08T21:00:16.0561994Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0381s] [ 85%] 2024-08-08T21:00:16.0563528Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0257s] [ 85%] 2024-08-08T21:00:16.0565060Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0256s] [ 85%] 2024-08-08T21:00:16.0566599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0481s] [ 85%] 2024-08-08T21:00:16.0568125Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0479s] [ 85%] 2024-08-08T21:00:16.0569752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0345s] [ 85%] 2024-08-08T21:00:16.0571362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1954s] [ 85%] 2024-08-08T21:00:16.0572875Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0273s] [ 85%] 2024-08-08T21:00:16.0574417Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0276s] [ 85%] 2024-08-08T21:00:16.0575932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0178s] [ 85%] 2024-08-08T21:00:16.0577482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0178s] [ 85%] 2024-08-08T21:00:16.0578999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0386s] [ 85%] 2024-08-08T21:00:16.0580606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0388s] [ 85%] 2024-08-08T21:00:16.0582193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0234s] [ 85%] 2024-08-08T21:00:16.0583740Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0238s] [ 85%] 2024-08-08T21:00:16.0585243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0271s] [ 85%] 2024-08-08T21:00:16.0586775Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0274s] [ 85%] 2024-08-08T21:00:16.0588317Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0181s] [ 85%] 2024-08-08T21:00:16.0589841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0181s] [ 85%] 2024-08-08T21:00:16.0591374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0377s] [ 85%] 2024-08-08T21:00:16.0592947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0383s] [ 85%] 2024-08-08T21:00:16.0594526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0238s] [ 85%] 2024-08-08T21:00:16.0596045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0240s] [ 85%] 2024-08-08T21:00:16.0597584Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0522s] [ 85%] 2024-08-08T21:00:16.0599121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0548s] [ 85%] 2024-08-08T21:00:16.0600668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0326s] [ 85%] 2024-08-08T21:00:16.0602203Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0334s] [ 85%] 2024-08-08T21:00:16.0603772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.2447s] [ 85%] 2024-08-08T21:00:16.0605381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0766s] [ 85%] 2024-08-08T21:00:16.0606929Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0448s] [ 85%] 2024-08-08T21:00:16.0608489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0446s] [ 85%] 2024-08-08T21:00:16.0610010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0504s] [ 85%] 2024-08-08T21:00:16.0611563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0519s] [ 85%] 2024-08-08T21:00:16.0613076Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0333s] [ 85%] 2024-08-08T21:00:16.0614615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0335s] [ 85%] 2024-08-08T21:00:16.0616176Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0699s] [ 85%] 2024-08-08T21:00:16.0617796Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0716s] [ 85%] 2024-08-08T21:00:16.0619311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.1992s] [ 85%] 2024-08-08T21:00:16.0620835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0442s] [ 85%] 2024-08-08T21:00:16.0622376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0563s] [ 85%] 2024-08-08T21:00:16.0623921Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0599s] [ 85%] 2024-08-08T21:00:16.0625460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0367s] [ 85%] 2024-08-08T21:00:16.0627030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0368s] [ 85%] 2024-08-08T21:00:16.0628626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0794s] [ 85%] 2024-08-08T21:00:16.0630166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0810s] [ 85%] 2024-08-08T21:00:16.0631726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0488s] [ 85%] 2024-08-08T21:00:16.0633276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.2019s] [ 85%] 2024-08-08T21:00:16.0634827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0549s] [ 85%] 2024-08-08T21:00:16.0636355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0576s] [ 85%] 2024-08-08T21:00:16.0637868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0375s] [ 85%] 2024-08-08T21:00:16.0639454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0369s] [ 85%] 2024-08-08T21:00:16.0641030Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0748s] [ 85%] 2024-08-08T21:00:16.0642585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0758s] [ 85%] 2024-08-08T21:00:16.0644109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0482s] [ 85%] 2024-08-08T21:00:16.0645668Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0486s] [ 85%] 2024-08-08T21:00:16.0647194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.2220s] [ 85%] 2024-08-08T21:00:16.0648738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0716s] [ 85%] 2024-08-08T21:00:16.0650298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0440s] [ 85%] 2024-08-08T21:00:16.0652540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0438s] [ 85%] 2024-08-08T21:00:16.0654094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0905s] [ 85%] 2024-08-08T21:00:16.0655681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0904s] [ 85%] 2024-08-08T21:00:16.0657251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0641s] [ 85%] 2024-08-08T21:00:16.0659172Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0628s] [ 85%] 2024-08-08T21:00:16.0660739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.2223s] [ 86%] 2024-08-08T21:00:16.0662265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0679s] [ 86%] 2024-08-08T21:00:16.0663892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0445s] [ 86%] 2024-08-08T21:00:16.0665490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0448s] [ 86%] 2024-08-08T21:00:16.0667009Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0858s] [ 86%] 2024-08-08T21:00:16.0668560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0860s] [ 86%] 2024-08-08T21:00:16.0670086Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0614s] [ 86%] 2024-08-08T21:00:16.0671636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0616s] [ 86%] 2024-08-08T21:00:16.0673148Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.2118s] [ 86%] 2024-08-08T21:00:16.0674797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0537s] [ 86%] 2024-08-08T21:00:16.0676388Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0302s] [ 86%] 2024-08-08T21:00:16.0677942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0304s] [ 86%] 2024-08-08T21:00:16.0679467Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0730s] [ 86%] 2024-08-08T21:00:16.0681023Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0740s] [ 86%] 2024-08-08T21:00:16.0682550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0428s] [ 86%] 2024-08-08T21:00:16.0684081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0427s] [ 86%] 2024-08-08T21:00:16.0685606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0490s] [ 86%] 2024-08-08T21:00:16.0687183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0488s] [ 86%] 2024-08-08T21:00:16.0688765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0314s] [ 86%] 2024-08-08T21:00:16.0690292Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0314s] [ 86%] 2024-08-08T21:00:16.0691835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.2321s] [ 86%] 2024-08-08T21:00:16.0693371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0704s] [ 86%] 2024-08-08T21:00:16.0694936Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0423s] [ 86%] 2024-08-08T21:00:16.0696434Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0428s] [ 86%] 2024-08-08T21:00:16.0698013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0123s] [ 86%] 2024-08-08T21:00:16.0699602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0123s] [ 86%] 2024-08-08T21:00:16.0701134Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0093s] [ 86%] 2024-08-08T21:00:16.0702680Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 86%] 2024-08-08T21:00:16.0704210Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0163s] [ 86%] 2024-08-08T21:00:16.0705770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0164s] [ 86%] 2024-08-08T21:00:16.0707286Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0102s] [ 86%] 2024-08-08T21:00:16.0708832Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 86%] 2024-08-08T21:00:16.0710406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0121s] [ 86%] 2024-08-08T21:00:16.0712005Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0121s] [ 86%] 2024-08-08T21:00:16.0713518Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0092s] [ 86%] 2024-08-08T21:00:16.0715041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 86%] 2024-08-08T21:00:16.0716578Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0161s] [ 86%] 2024-08-08T21:00:16.0718231Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0162s] [ 86%] 2024-08-08T21:00:16.0719647Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0100s] [ 86%] 2024-08-08T21:00:16.0721216Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0104s] [ 86%] 2024-08-08T21:00:16.0722798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0144s] [ 86%] 2024-08-08T21:00:16.0724336Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0140s] [ 86%] 2024-08-08T21:00:16.0725872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0099s] [ 86%] 2024-08-08T21:00:16.0727406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0098s] [ 86%] 2024-08-08T21:00:16.0728957Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0178s] [ 86%] 2024-08-08T21:00:16.0730491Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0179s] [ 86%] 2024-08-08T21:00:16.0732010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0109s] [ 86%] 2024-08-08T21:00:16.0733601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0110s] [ 86%] 2024-08-08T21:00:16.0735160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0143s] [ 86%] 2024-08-08T21:00:16.0736714Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0149s] [ 86%] 2024-08-08T21:00:16.0738217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0100s] [ 86%] 2024-08-08T21:00:16.0739759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 86%] 2024-08-08T21:00:16.0741283Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0171s] [ 86%] 2024-08-08T21:00:16.0742828Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0171s] [ 86%] 2024-08-08T21:00:16.0744391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0104s] [ 86%] 2024-08-08T21:00:16.0745972Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0103s] [ 86%] 2024-08-08T21:00:16.0747504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0174s] [ 86%] 2024-08-08T21:00:16.0749031Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0178s] [ 86%] 2024-08-08T21:00:16.0750572Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0125s] [ 86%] 2024-08-08T21:00:16.0752112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0125s] [ 86%] 2024-08-08T21:00:16.0753654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0211s] [ 86%] 2024-08-08T21:00:16.0755223Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0211s] [ 86%] 2024-08-08T21:00:16.0756815Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0133s] [ 86%] 2024-08-08T21:00:16.0758713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0133s] [ 86%] 2024-08-08T21:00:16.0760281Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0177s] [ 86%] 2024-08-08T21:00:16.0761803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0180s] [ 86%] 2024-08-08T21:00:16.0763320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.1796s] [ 86%] 2024-08-08T21:00:16.0764871Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0130s] [ 86%] 2024-08-08T21:00:16.0766385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0213s] [ 86%] 2024-08-08T21:00:16.0768021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0215s] [ 86%] 2024-08-08T21:00:16.0769615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0140s] [ 86%] 2024-08-08T21:00:16.0771162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0140s] [ 86%] 2024-08-08T21:00:16.0772667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0114s] [ 86%] 2024-08-08T21:00:16.0774211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0115s] [ 86%] 2024-08-08T21:00:16.0775725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0084s] [ 86%] 2024-08-08T21:00:16.0777251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 86%] 2024-08-08T21:00:16.0778782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0153s] [ 86%] 2024-08-08T21:00:16.0780374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0155s] [ 86%] 2024-08-08T21:00:16.0781975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0093s] [ 87%] 2024-08-08T21:00:16.0783496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0094s] [ 87%] 2024-08-08T21:00:16.0785033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0116s] [ 87%] 2024-08-08T21:00:16.0786531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0117s] [ 87%] 2024-08-08T21:00:16.0788065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0085s] [ 87%] 2024-08-08T21:00:16.0789583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0085s] [ 87%] 2024-08-08T21:00:16.0791153Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0157s] [ 87%] 2024-08-08T21:00:16.0792737Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0159s] [ 87%] 2024-08-08T21:00:16.0794250Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0096s] [ 87%] 2024-08-08T21:00:16.0795788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0097s] [ 87%] 2024-08-08T21:00:16.0797313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.1014s] [ 87%] 2024-08-08T21:00:16.0798872Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1079s] [ 87%] 2024-08-08T21:00:16.0800392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0590s] [ 87%] 2024-08-08T21:00:16.0801945Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0590s] [ 87%] 2024-08-08T21:00:16.0803475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.3084s] [ 87%] 2024-08-08T21:00:16.0805082Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1546s] [ 87%] 2024-08-08T21:00:16.0806669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0842s] [ 87%] 2024-08-08T21:00:16.0808213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0847s] [ 87%] 2024-08-08T21:00:16.0809758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0873s] [ 87%] 2024-08-08T21:00:16.0811298Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.2375s] [ 87%] 2024-08-08T21:00:16.0812837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0609s] [ 87%] 2024-08-08T21:00:16.0814473Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0602s] [ 87%] 2024-08-08T21:00:16.0816000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1200s] [ 87%] 2024-08-08T21:00:16.0817531Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1214s] [ 87%] 2024-08-08T21:00:16.0819064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.2208s] [ 87%] 2024-08-08T21:00:16.0820593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0805s] [ 87%] 2024-08-08T21:00:16.0822160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.1103s] [ 87%] 2024-08-08T21:00:16.0823664Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1185s] [ 87%] 2024-08-08T21:00:16.0825180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0692s] [ 87%] 2024-08-08T21:00:16.0826776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.2271s] [ 87%] 2024-08-08T21:00:16.0828357Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1560s] [ 87%] 2024-08-08T21:00:16.0829915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1569s] [ 87%] 2024-08-08T21:00:16.0831451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0922s] [ 87%] 2024-08-08T21:00:16.0833022Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.2263s] [ 87%] 2024-08-08T21:00:16.0834551Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0931s] [ 87%] 2024-08-08T21:00:16.0836094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0966s] [ 87%] 2024-08-08T21:00:16.0837646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0681s] [ 87%] 2024-08-08T21:00:16.0839247Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0696s] [ 87%] 2024-08-08T21:00:16.0840772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.2868s] [ 87%] 2024-08-08T21:00:16.0842303Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1281s] [ 87%] 2024-08-08T21:00:16.0843840Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0890s] [ 87%] 2024-08-08T21:00:16.0845375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0884s] [ 87%] 2024-08-08T21:00:16.0846912Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.2707s] [ 87%] 2024-08-08T21:00:16.0848438Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1354s] [ 87%] 2024-08-08T21:00:16.0850016Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0794s] [ 87%] 2024-08-08T21:00:16.0851603Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0799s] [ 87%] 2024-08-08T21:00:16.0853143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.3043s] [ 87%] 2024-08-08T21:00:16.0854727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1735s] [ 87%] 2024-08-08T21:00:16.0856276Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.1188s] [ 87%] 2024-08-08T21:00:16.0857845Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1214s] [ 87%] 2024-08-08T21:00:16.0859748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.2496s] [ 87%] 2024-08-08T21:00:16.0861416Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1138s] [ 87%] 2024-08-08T21:00:16.0863010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0799s] [ 87%] 2024-08-08T21:00:16.0864567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0797s] [ 87%] 2024-08-08T21:00:16.0866094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.2719s] [ 87%] 2024-08-08T21:00:16.0867654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1437s] [ 87%] 2024-08-08T21:00:16.0869186Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.1091s] [ 87%] 2024-08-08T21:00:16.0870757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1085s] [ 87%] 2024-08-08T21:00:16.0872254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.2482s] [ 87%] 2024-08-08T21:00:16.0873842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1014s] [ 87%] 2024-08-08T21:00:16.0875457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0557s] [ 87%] 2024-08-08T21:00:16.0876980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0582s] [ 87%] 2024-08-08T21:00:16.0878517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1426s] [ 87%] 2024-08-08T21:00:16.0880056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1452s] [ 87%] 2024-08-08T21:00:16.0881601Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.2380s] [ 87%] 2024-08-08T21:00:16.0883135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0814s] [ 87%] 2024-08-08T21:00:16.0884722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0828s] [ 87%] 2024-08-08T21:00:16.0886304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0852s] [ 87%] 2024-08-08T21:00:16.0887829Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0588s] [ 87%] 2024-08-08T21:00:16.0889378Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0575s] [ 87%] 2024-08-08T21:00:16.0890902Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.2718s] [ 87%] 2024-08-08T21:00:16.0892460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1167s] [ 87%] 2024-08-08T21:00:16.0893978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0771s] [ 87%] 2024-08-08T21:00:16.0895519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0777s] [ 87%] 2024-08-08T21:00:16.0897084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0168s] [ 87%] 2024-08-08T21:00:16.0898683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0173s] [ 87%] 2024-08-08T21:00:16.0900189Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0121s] [ 87%] 2024-08-08T21:00:16.0901732Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0119s] [ 87%] 2024-08-08T21:00:16.0903263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0225s] [ 88%] 2024-08-08T21:00:16.0904808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0225s] [ 88%] 2024-08-08T21:00:16.0906345Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0152s] [ 88%] 2024-08-08T21:00:16.0907917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1773s] [ 88%] 2024-08-08T21:00:16.0909496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0171s] [ 88%] 2024-08-08T21:00:16.0911040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0176s] [ 88%] 2024-08-08T21:00:16.0912579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0124s] [ 88%] 2024-08-08T21:00:16.0914113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0122s] [ 88%] 2024-08-08T21:00:16.0915660Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0230s] [ 88%] 2024-08-08T21:00:16.0917194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0228s] [ 88%] 2024-08-08T21:00:16.0918722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0156s] [ 88%] 2024-08-08T21:00:16.0920293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0155s] [ 88%] 2024-08-08T21:00:16.0921866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0187s] [ 88%] 2024-08-08T21:00:16.0923404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0190s] [ 88%] 2024-08-08T21:00:16.0924915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0135s] [ 88%] 2024-08-08T21:00:16.0926464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0135s] [ 88%] 2024-08-08T21:00:16.0927997Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0241s] [ 88%] 2024-08-08T21:00:16.0929544Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0247s] [ 88%] 2024-08-08T21:00:16.0931102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0164s] [ 88%] 2024-08-08T21:00:16.0932700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0168s] [ 88%] 2024-08-08T21:00:16.0934211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0193s] [ 88%] 2024-08-08T21:00:16.0935739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0194s] [ 88%] 2024-08-08T21:00:16.0937278Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0141s] [ 88%] 2024-08-08T21:00:16.0938820Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0142s] [ 88%] 2024-08-08T21:00:16.0940366Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0249s] [ 88%] 2024-08-08T21:00:16.0941893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0248s] [ 88%] 2024-08-08T21:00:16.0943502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0168s] [ 88%] 2024-08-08T21:00:16.0945046Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0168s] [ 88%] 2024-08-08T21:00:16.0946571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0224s] [ 88%] 2024-08-08T21:00:16.0948116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0227s] [ 88%] 2024-08-08T21:00:16.0949621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0166s] [ 88%] 2024-08-08T21:00:16.0951160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0168s] [ 88%] 2024-08-08T21:00:16.0952685Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.1994s] [ 88%] 2024-08-08T21:00:16.0954275Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0280s] [ 88%] 2024-08-08T21:00:16.0955897Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0206s] [ 88%] 2024-08-08T21:00:16.0957460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0215s] [ 88%] 2024-08-08T21:00:16.0959328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0224s] [ 88%] 2024-08-08T21:00:16.0960913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0233s] [ 88%] 2024-08-08T21:00:16.0962439Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0165s] [ 88%] 2024-08-08T21:00:16.0963990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0168s] [ 88%] 2024-08-08T21:00:16.0965503Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0284s] [ 88%] 2024-08-08T21:00:16.0967116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0283s] [ 88%] 2024-08-08T21:00:16.0968734Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0216s] [ 88%] 2024-08-08T21:00:16.0970255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0213s] [ 88%] 2024-08-08T21:00:16.0971802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0157s] [ 88%] 2024-08-08T21:00:16.0973316Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0163s] [ 88%] 2024-08-08T21:00:16.0974861Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0114s] [ 88%] 2024-08-08T21:00:16.0976394Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 88%] 2024-08-08T21:00:16.0977993Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0217s] [ 88%] 2024-08-08T21:00:16.0979588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0219s] [ 88%] 2024-08-08T21:00:16.0981127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0148s] [ 88%] 2024-08-08T21:00:16.0982651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0146s] [ 88%] 2024-08-08T21:00:16.0984157Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0162s] [ 88%] 2024-08-08T21:00:16.0985711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0162s] [ 88%] 2024-08-08T21:00:16.0987225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0116s] [ 88%] 2024-08-08T21:00:16.0988771Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0115s] [ 88%] 2024-08-08T21:00:16.0990327Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0221s] [ 88%] 2024-08-08T21:00:16.0991927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0222s] [ 88%] 2024-08-08T21:00:16.0993436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0150s] [ 88%] 2024-08-08T21:00:16.0994974Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.1832s] [ 88%] 2024-08-08T21:00:16.0996494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0125s] [ 88%] 2024-08-08T21:00:16.0998033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0123s] [ 88%] 2024-08-08T21:00:16.0999567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0093s] [ 88%] 2024-08-08T21:00:16.1001136Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0091s] [ 88%] 2024-08-08T21:00:16.1002720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0168s] [ 88%] 2024-08-08T21:00:16.1004269Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0168s] [ 88%] 2024-08-08T21:00:16.1005781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0103s] [ 88%] 2024-08-08T21:00:16.1007305Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0105s] [ 88%] 2024-08-08T21:00:16.1008831Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0126s] [ 88%] 2024-08-08T21:00:16.1010371Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0125s] [ 88%] 2024-08-08T21:00:16.1011877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0091s] [ 88%] 2024-08-08T21:00:16.1013449Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0093s] [ 88%] 2024-08-08T21:00:16.1015018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0168s] [ 88%] 2024-08-08T21:00:16.1016567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0172s] [ 88%] 2024-08-08T21:00:16.1018143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0107s] [ 88%] 2024-08-08T21:00:16.1019630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0109s] [ 88%] 2024-08-08T21:00:16.1021150Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0146s] [ 88%] 2024-08-08T21:00:16.1022697Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0149s] [ 89%] 2024-08-08T21:00:16.1024251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0104s] [ 89%] 2024-08-08T21:00:16.1025857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0102s] [ 89%] 2024-08-08T21:00:16.1027372Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0186s] [ 89%] 2024-08-08T21:00:16.1028905Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0185s] [ 89%] 2024-08-08T21:00:16.1030437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0111s] [ 89%] 2024-08-08T21:00:16.1031969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 89%] 2024-08-08T21:00:16.1033493Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0144s] [ 89%] 2024-08-08T21:00:16.1035015Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0145s] [ 89%] 2024-08-08T21:00:16.1036550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0099s] [ 89%] 2024-08-08T21:00:16.1038142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0100s] [ 89%] 2024-08-08T21:00:16.1039781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0185s] [ 89%] 2024-08-08T21:00:16.1041304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0186s] [ 89%] 2024-08-08T21:00:16.1042838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0112s] [ 89%] 2024-08-08T21:00:16.1044375Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0107s] [ 89%] 2024-08-08T21:00:16.1045884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0181s] [ 89%] 2024-08-08T21:00:16.1047432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0182s] [ 89%] 2024-08-08T21:00:16.1048992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0127s] [ 89%] 2024-08-08T21:00:16.1050594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0129s] [ 89%] 2024-08-08T21:00:16.1052110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0219s] [ 89%] 2024-08-08T21:00:16.1053653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0219s] [ 89%] 2024-08-08T21:00:16.1055213Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0135s] [ 89%] 2024-08-08T21:00:16.1056758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0134s] [ 89%] 2024-08-08T21:00:16.1058255Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0178s] [ 89%] 2024-08-08T21:00:16.1060156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0182s] [ 89%] 2024-08-08T21:00:16.1061779Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0128s] [ 89%] 2024-08-08T21:00:16.1063382Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0128s] [ 89%] 2024-08-08T21:00:16.1064918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0219s] [ 89%] 2024-08-08T21:00:16.1066455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0219s] [ 89%] 2024-08-08T21:00:16.1068002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0139s] [ 89%] 2024-08-08T21:00:16.1069530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0137s] [ 89%] 2024-08-08T21:00:16.1071059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0113s] [ 89%] 2024-08-08T21:00:16.1072652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0114s] [ 89%] 2024-08-08T21:00:16.1074259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 89%] 2024-08-08T21:00:16.1075782Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 89%] 2024-08-08T21:00:16.1077301Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0149s] [ 89%] 2024-08-08T21:00:16.1078858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0152s] [ 89%] 2024-08-08T21:00:16.1080376Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0090s] [ 89%] 2024-08-08T21:00:16.1081916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 89%] 2024-08-08T21:00:16.1083423Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0113s] [ 89%] 2024-08-08T21:00:16.1085014Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0111s] [ 89%] 2024-08-08T21:00:16.1086565Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0085s] [ 89%] 2024-08-08T21:00:16.1088098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0086s] [ 89%] 2024-08-08T21:00:16.1089616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0154s] [ 89%] 2024-08-08T21:00:16.1091166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0151s] [ 89%] 2024-08-08T21:00:16.1092678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0097s] [ 89%] 2024-08-08T21:00:16.1094199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_512_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0089s] [ 89%] 2024-08-08T21:00:16.1095785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0072s] [ 89%] 2024-08-08T21:00:16.1097384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 89%] 2024-08-08T21:00:16.1098917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 89%] 2024-08-08T21:00:16.1100450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 89%] 2024-08-08T21:00:16.1102002Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0085s] [ 89%] 2024-08-08T21:00:16.1103540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0084s] [ 89%] 2024-08-08T21:00:16.1105078Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 89%] 2024-08-08T21:00:16.1106614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 89%] 2024-08-08T21:00:16.1108175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0068s] [ 89%] 2024-08-08T21:00:16.1109798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 89%] 2024-08-08T21:00:16.1111295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 89%] 2024-08-08T21:00:16.1112852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 89%] 2024-08-08T21:00:16.1114384Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0076s] [ 89%] 2024-08-08T21:00:16.1115933Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 89%] 2024-08-08T21:00:16.1117447Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 89%] 2024-08-08T21:00:16.1119047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 89%] 2024-08-08T21:00:16.1120639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0074s] [ 89%] 2024-08-08T21:00:16.1122180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 89%] 2024-08-08T21:00:16.1123695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 89%] 2024-08-08T21:00:16.1125229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 89%] 2024-08-08T21:00:16.1126768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0090s] [ 89%] 2024-08-08T21:00:16.1128322Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 89%] 2024-08-08T21:00:16.1129859Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0071s] [ 89%] 2024-08-08T21:00:16.1131419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 89%] 2024-08-08T21:00:16.1132995Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0073s] [ 89%] 2024-08-08T21:00:16.1134526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 89%] 2024-08-08T21:00:16.1136065Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0068s] [ 89%] 2024-08-08T21:00:16.1137606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 89%] 2024-08-08T21:00:16.1139140Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0084s] [ 89%] 2024-08-08T21:00:16.1140687Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0079s] [ 89%] 2024-08-08T21:00:16.1142251Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0078s] [ 89%] 2024-08-08T21:00:16.1143858Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 90%] 2024-08-08T21:00:16.1145370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0088s] [ 90%] 2024-08-08T21:00:16.1146915Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0092s] [ 90%] 2024-08-08T21:00:16.1148430Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0071s] [ 90%] 2024-08-08T21:00:16.1149973Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 90%] 2024-08-08T21:00:16.1151570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0099s] [ 90%] 2024-08-08T21:00:16.1153054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0100s] [ 90%] 2024-08-08T21:00:16.1154624Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0082s] [ 90%] 2024-08-08T21:00:16.1156248Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 90%] 2024-08-08T21:00:16.1157797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0082s] [ 90%] 2024-08-08T21:00:16.1161406Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 90%] 2024-08-08T21:00:16.1162942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 90%] 2024-08-08T21:00:16.1164472Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 90%] 2024-08-08T21:00:16.1166013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0096s] [ 90%] 2024-08-08T21:00:16.1167653Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0089s] [ 90%] 2024-08-08T21:00:16.1169267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0090s] [ 90%] 2024-08-08T21:00:16.1170788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0090s] [ 90%] 2024-08-08T21:00:16.1172302Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0067s] [ 90%] 2024-08-08T21:00:16.1173864Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 90%] 2024-08-08T21:00:16.1175383Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 90%] 2024-08-08T21:00:16.1176930Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 90%] 2024-08-08T21:00:16.1178455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0083s] [ 90%] 2024-08-08T21:00:16.1180066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 90%] 2024-08-08T21:00:16.1181644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 90%] 2024-08-08T21:00:16.1183184Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 90%] 2024-08-08T21:00:16.1184692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0064s] [ 90%] 2024-08-08T21:00:16.1186244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 90%] 2024-08-08T21:00:16.1187758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 90%] 2024-08-08T21:00:16.1189277Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 90%] 2024-08-08T21:00:16.1190860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 90%] 2024-08-08T21:00:16.1192443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 90%] 2024-08-08T21:00:16.1193978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0068s] [ 90%] 2024-08-08T21:00:16.1195534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 90%] 2024-08-08T21:00:16.1197052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0101s] [ 90%] 2024-08-08T21:00:16.1198588Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0112s] [ 90%] 2024-08-08T21:00:16.1200111Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0075s] [ 90%] 2024-08-08T21:00:16.1201636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 90%] 2024-08-08T21:00:16.1203208Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0132s] [ 90%] 2024-08-08T21:00:16.1204830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0136s] [ 90%] 2024-08-08T21:00:16.1206347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0095s] [ 90%] 2024-08-08T21:00:16.1207896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0094s] [ 90%] 2024-08-08T21:00:16.1209389Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0088s] [ 90%] 2024-08-08T21:00:16.1210941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0087s] [ 90%] 2024-08-08T21:00:16.1212458Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0081s] [ 90%] 2024-08-08T21:00:16.1214039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 90%] 2024-08-08T21:00:16.1215607Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0104s] [ 90%] 2024-08-08T21:00:16.1217151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0102s] [ 90%] 2024-08-08T21:00:16.1218686Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0091s] [ 90%] 2024-08-08T21:00:16.1220194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0091s] [ 90%] 2024-08-08T21:00:16.1221726Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0112s] [ 90%] 2024-08-08T21:00:16.1223320Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0114s] [ 90%] 2024-08-08T21:00:16.1224783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0082s] [ 90%] 2024-08-08T21:00:16.1226368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 90%] 2024-08-08T21:00:16.1227943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0138s] [ 90%] 2024-08-08T21:00:16.1229474Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0143s] [ 90%] 2024-08-08T21:00:16.1231011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0097s] [ 90%] 2024-08-08T21:00:16.1232547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0101s] [ 90%] 2024-08-08T21:00:16.1234081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0090s] [ 90%] 2024-08-08T21:00:16.1235609Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0094s] [ 90%] 2024-08-08T21:00:16.1237181Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0087s] [ 90%] 2024-08-08T21:00:16.1238772Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 90%] 2024-08-08T21:00:16.1240294Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0109s] [ 90%] 2024-08-08T21:00:16.1241838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0110s] [ 90%] 2024-08-08T21:00:16.1243351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0099s] [ 90%] 2024-08-08T21:00:16.1244896Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0097s] [ 90%] 2024-08-08T21:00:16.1246405Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0133s] [ 90%] 2024-08-08T21:00:16.1247947Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0137s] [ 90%] 2024-08-08T21:00:16.1249502Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0100s] [ 90%] 2024-08-08T21:00:16.1251066Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0100s] [ 90%] 2024-08-08T21:00:16.1252605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0164s] [ 90%] 2024-08-08T21:00:16.1254135Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0161s] [ 90%] 2024-08-08T21:00:16.1255724Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0124s] [ 90%] 2024-08-08T21:00:16.1257263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0128s] [ 90%] 2024-08-08T21:00:16.1259222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0106s] [ 90%] 2024-08-08T21:00:16.1260886Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0108s] [ 90%] 2024-08-08T21:00:16.1262504Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0093s] [ 90%] 2024-08-08T21:00:16.1264025Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 90%] 2024-08-08T21:00:16.1265560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0120s] [ 90%] 2024-08-08T21:00:16.1267091Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0121s] [ 91%] 2024-08-08T21:00:16.1268616Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0113s] [ 91%] 2024-08-08T21:00:16.1271667Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 91%] 2024-08-08T21:00:16.1274657Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0098s] [ 91%] 2024-08-08T21:00:16.1277727Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0100s] [ 91%] 2024-08-08T21:00:16.1280848Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0071s] [ 91%] 2024-08-08T21:00:16.1283811Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 91%] 2024-08-08T21:00:16.1286821Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0126s] [ 91%] 2024-08-08T21:00:16.1289770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0128s] [ 91%] 2024-08-08T21:00:16.1292785Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0088s] [ 91%] 2024-08-08T21:00:16.1295760Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0088s] [ 91%] 2024-08-08T21:00:16.1298793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0084s] [ 91%] 2024-08-08T21:00:16.1301826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 91%] 2024-08-08T21:00:16.1304767Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0074s] [ 91%] 2024-08-08T21:00:16.1307735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 91%] 2024-08-08T21:00:16.1310713Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0095s] [ 91%] 2024-08-08T21:00:16.1313707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0097s] [ 91%] 2024-08-08T21:00:16.1316698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0085s] [ 91%] 2024-08-08T21:00:16.1319651Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0084s] [ 91%] 2024-08-08T21:00:16.1322683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 91%] 2024-08-08T21:00:16.1325718Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 91%] 2024-08-08T21:00:16.1328682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 91%] 2024-08-08T21:00:16.1331662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 91%] 2024-08-08T21:00:16.1334655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 91%] 2024-08-08T21:00:16.1337630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 91%] 2024-08-08T21:00:16.1340602Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 91%] 2024-08-08T21:00:16.1343645Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 91%] 2024-08-08T21:00:16.1346659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 91%] 2024-08-08T21:00:16.1349621Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 91%] 2024-08-08T21:00:16.1352571Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 91%] 2024-08-08T21:00:16.1355579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 91%] 2024-08-08T21:00:16.1358978Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 91%] 2024-08-08T21:00:16.1361966Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 91%] 2024-08-08T21:00:16.1364941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 91%] 2024-08-08T21:00:16.1368010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 91%] 2024-08-08T21:00:16.1371051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 91%] 2024-08-08T21:00:16.1374037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 91%] 2024-08-08T21:00:16.1377033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 91%] 2024-08-08T21:00:16.1379999Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 91%] 2024-08-08T21:00:16.1382977Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 91%] 2024-08-08T21:00:16.1385967Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 91%] 2024-08-08T21:00:16.1389011Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 91%] 2024-08-08T21:00:16.1392079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 91%] 2024-08-08T21:00:16.1395051Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 91%] 2024-08-08T21:00:16.1398024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 91%] 2024-08-08T21:00:16.1400979Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 91%] 2024-08-08T21:00:16.1403932Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 91%] 2024-08-08T21:00:16.1406893Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 91%] 2024-08-08T21:00:16.1409873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 91%] 2024-08-08T21:00:16.1412892Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 91%] 2024-08-08T21:00:16.1415918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 91%] 2024-08-08T21:00:16.1418882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 91%] 2024-08-08T21:00:16.1421841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 91%] 2024-08-08T21:00:16.1424817Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 91%] 2024-08-08T21:00:16.1427798Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 91%] 2024-08-08T21:00:16.1430777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 91%] 2024-08-08T21:00:16.1433819Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 91%] 2024-08-08T21:00:16.1436838Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 91%] 2024-08-08T21:00:16.1439826Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 91%] 2024-08-08T21:00:16.1442795Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 91%] 2024-08-08T21:00:16.1445768Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 91%] 2024-08-08T21:00:16.1448741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 91%] 2024-08-08T21:00:16.1451699Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 91%] 2024-08-08T21:00:16.1454649Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0072s] [ 91%] 2024-08-08T21:00:16.1457736Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 91%] 2024-08-08T21:00:16.1461196Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 91%] 2024-08-08T21:00:16.1464166Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 91%] 2024-08-08T21:00:16.1467138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 91%] 2024-08-08T21:00:16.1470100Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 91%] 2024-08-08T21:00:16.1473061Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 91%] 2024-08-08T21:00:16.1476027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 91%] 2024-08-08T21:00:16.1479085Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 91%] 2024-08-08T21:00:16.1482143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 91%] 2024-08-08T21:00:16.1485132Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 91%] 2024-08-08T21:00:16.1488080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 91%] 2024-08-08T21:00:16.1491048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 91%] 2024-08-08T21:00:16.1494018Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 91%] 2024-08-08T21:00:16.1496954Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 91%] 2024-08-08T21:00:16.1499904Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 92%] 2024-08-08T21:00:16.1502914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 92%] 2024-08-08T21:00:16.1505959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 92%] 2024-08-08T21:00:16.1508916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 92%] 2024-08-08T21:00:16.1511877Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 92%] 2024-08-08T21:00:16.1514863Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0172s] [ 92%] 2024-08-08T21:00:16.1517867Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0182s] [ 92%] 2024-08-08T21:00:16.1520830Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0111s] [ 92%] 2024-08-08T21:00:16.1523868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0111s] [ 92%] 2024-08-08T21:00:16.1526925Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0228s] [ 92%] 2024-08-08T21:00:16.1529934Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0231s] [ 92%] 2024-08-08T21:00:16.1532935Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0143s] [ 92%] 2024-08-08T21:00:16.1535914Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0143s] [ 92%] 2024-08-08T21:00:16.1538926Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0126s] [ 92%] 2024-08-08T21:00:16.1541895Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0125s] [ 92%] 2024-08-08T21:00:16.1544866Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0117s] [ 92%] 2024-08-08T21:00:16.1547968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 92%] 2024-08-08T21:00:16.1551017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0144s] [ 92%] 2024-08-08T21:00:16.1554001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0147s] [ 92%] 2024-08-08T21:00:16.1557034Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0136s] [ 92%] 2024-08-08T21:00:16.1560396Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0137s] [ 92%] 2024-08-08T21:00:16.1563402Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0189s] [ 92%] 2024-08-08T21:00:16.1566415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.1917s] [ 92%] 2024-08-08T21:00:16.1569470Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0130s] [ 92%] 2024-08-08T21:00:16.1572511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0131s] [ 92%] 2024-08-08T21:00:16.1575520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0246s] [ 92%] 2024-08-08T21:00:16.1578517Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0252s] [ 92%] 2024-08-08T21:00:16.1581574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0166s] [ 92%] 2024-08-08T21:00:16.1584662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0165s] [ 92%] 2024-08-08T21:00:16.1587630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0145s] [ 92%] 2024-08-08T21:00:16.1590615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0146s] [ 92%] 2024-08-08T21:00:16.1593659Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0135s] [ 92%] 2024-08-08T21:00:16.1596719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0141s] [ 92%] 2024-08-08T21:00:16.1599710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0166s] [ 92%] 2024-08-08T21:00:16.1602698Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0165s] [ 92%] 2024-08-08T21:00:16.1605674Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0153s] [ 92%] 2024-08-08T21:00:16.1608678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0152s] [ 92%] 2024-08-08T21:00:16.1611655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0233s] [ 92%] 2024-08-08T21:00:16.1614694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0243s] [ 92%] 2024-08-08T21:00:16.1617738Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0165s] [ 92%] 2024-08-08T21:00:16.1620710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0166s] [ 92%] 2024-08-08T21:00:16.1623700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0285s] [ 92%] 2024-08-08T21:00:16.1626702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0290s] [ 92%] 2024-08-08T21:00:16.1629711Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0219s] [ 92%] 2024-08-08T21:00:16.1632719Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0218s] [ 92%] 2024-08-08T21:00:16.1635695Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0174s] [ 92%] 2024-08-08T21:00:16.1638709Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0171s] [ 92%] 2024-08-08T21:00:16.1641742Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.1839s] [ 92%] 2024-08-08T21:00:16.1644716Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0164s] [ 92%] 2024-08-08T21:00:16.1647693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0199s] [ 92%] 2024-08-08T21:00:16.1650690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0197s] [ 92%] 2024-08-08T21:00:16.1653672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0182s] [ 92%] 2024-08-08T21:00:16.1656694Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0181s] [ 92%] 2024-08-08T21:00:16.1660121Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0168s] [ 92%] 2024-08-08T21:00:16.1670530Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0174s] [ 92%] 2024-08-08T21:00:16.1673543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0115s] [ 92%] 2024-08-08T21:00:16.1676537Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0112s] [ 92%] 2024-08-08T21:00:16.1679526Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0218s] [ 92%] 2024-08-08T21:00:16.1682542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0226s] [ 92%] 2024-08-08T21:00:16.1685514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0145s] [ 92%] 2024-08-08T21:00:16.1688498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0141s] [ 92%] 2024-08-08T21:00:16.1691604Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0126s] [ 92%] 2024-08-08T21:00:16.1694681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0127s] [ 92%] 2024-08-08T21:00:16.1697643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0114s] [ 92%] 2024-08-08T21:00:16.1700599Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0116s] [ 92%] 2024-08-08T21:00:16.1703581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0144s] [ 92%] 2024-08-08T21:00:16.1706606Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0144s] [ 92%] 2024-08-08T21:00:16.1709555Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0133s] [ 92%] 2024-08-08T21:00:16.1712583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0136s] [ 92%] 2024-08-08T21:00:16.1715639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 92%] 2024-08-08T21:00:16.1718615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 92%] 2024-08-08T21:00:16.1721596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0063s] [ 92%] 2024-08-08T21:00:16.1724574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 92%] 2024-08-08T21:00:16.1727678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 92%] 2024-08-08T21:00:16.1730561Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 92%] 2024-08-08T21:00:16.1733532Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0072s] [ 92%] 2024-08-08T21:00:16.1736524Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 92%] 2024-08-08T21:00:16.1739543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0067s] [ 92%] 2024-08-08T21:00:16.1742574Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 93%] 2024-08-08T21:00:16.1745542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 93%] 2024-08-08T21:00:16.1748520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 93%] 2024-08-08T21:00:16.1751482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 93%] 2024-08-08T21:00:16.1754466Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 93%] 2024-08-08T21:00:16.1757514Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 93%] 2024-08-08T21:00:16.1761098Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 93%] 2024-08-08T21:00:16.1764170Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 93%] 2024-08-08T21:00:16.1767133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 93%] 2024-08-08T21:00:16.1770138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 93%] 2024-08-08T21:00:16.1773107Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 93%] 2024-08-08T21:00:16.1776080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 93%] 2024-08-08T21:00:16.1779081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0071s] [ 93%] 2024-08-08T21:00:16.1782084Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 93%] 2024-08-08T21:00:16.1785108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 93%] 2024-08-08T21:00:16.1788144Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 93%] 2024-08-08T21:00:16.1791113Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 93%] 2024-08-08T21:00:16.1794087Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 93%] 2024-08-08T21:00:16.1797094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 93%] 2024-08-08T21:00:16.1800036Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 93%] 2024-08-08T21:00:16.1803006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 93%] 2024-08-08T21:00:16.1806037Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 93%] 2024-08-08T21:00:16.1809070Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 93%] 2024-08-08T21:00:16.1812052Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0064s] [ 93%] 2024-08-08T21:00:16.1815039Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 93%] 2024-08-08T21:00:16.1818007Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0065s] [ 93%] 2024-08-08T21:00:16.1820987Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 93%] 2024-08-08T21:00:16.1823968Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0079s] [ 93%] 2024-08-08T21:00:16.1826961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0077s] [ 93%] 2024-08-08T21:00:16.1830010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0073s] [ 93%] 2024-08-08T21:00:16.1833047Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0072s] [ 93%] 2024-08-08T21:00:16.1836010Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 93%] 2024-08-08T21:00:16.1838990Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 93%] 2024-08-08T21:00:16.1841960Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0064s] [ 93%] 2024-08-08T21:00:16.1844943Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0067s] [ 93%] 2024-08-08T21:00:16.1847908Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0081s] [ 93%] 2024-08-08T21:00:16.1850922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0082s] [ 93%] 2024-08-08T21:00:16.1853958Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 93%] 2024-08-08T21:00:16.1856989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0077s] [ 93%] 2024-08-08T21:00:16.1860374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 93%] 2024-08-08T21:00:16.1863370Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 93%] 2024-08-08T21:00:16.1866343Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 93%] 2024-08-08T21:00:16.1869295Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 93%] 2024-08-08T21:00:16.1872268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 93%] 2024-08-08T21:00:16.1875356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 93%] 2024-08-08T21:00:16.1878415Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 93%] 2024-08-08T21:00:16.1881395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 93%] 2024-08-08T21:00:16.1884344Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 93%] 2024-08-08T21:00:16.1887313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 93%] 2024-08-08T21:00:16.1890267Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 93%] 2024-08-08T21:00:16.1893222Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 93%] 2024-08-08T21:00:16.1896263Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0071s] [ 93%] 2024-08-08T21:00:16.1899311Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 93%] 2024-08-08T21:00:16.1902256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 93%] 2024-08-08T21:00:16.1905254Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 93%] 2024-08-08T21:00:16.1908191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 93%] 2024-08-08T21:00:16.1911182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 93%] 2024-08-08T21:00:16.1914149Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 93%] 2024-08-08T21:00:16.1917109Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 93%] 2024-08-08T21:00:16.1920146Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 93%] 2024-08-08T21:00:16.1923180Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 93%] 2024-08-08T21:00:16.1926159Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 93%] 2024-08-08T21:00:16.1929143Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 93%] 2024-08-08T21:00:16.1932112Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 93%] 2024-08-08T21:00:16.1935064Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 93%] 2024-08-08T21:00:16.1938024Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 93%] 2024-08-08T21:00:16.1941033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 93%] 2024-08-08T21:00:16.1944058Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 93%] 2024-08-08T21:00:16.1947035Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 93%] 2024-08-08T21:00:16.1949988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 93%] 2024-08-08T21:00:16.1952975Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 93%] 2024-08-08T21:00:16.1955992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 93%] 2024-08-08T21:00:16.1959351Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 93%] 2024-08-08T21:00:16.1962368Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 93%] 2024-08-08T21:00:16.1965448Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 93%] 2024-08-08T21:00:16.1968488Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 93%] 2024-08-08T21:00:16.1971479Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 93%] 2024-08-08T21:00:16.1974464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 93%] 2024-08-08T21:00:16.1977454Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 94%] 2024-08-08T21:00:16.1980419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 94%] 2024-08-08T21:00:16.1983373Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 94%] 2024-08-08T21:00:16.1986385Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 94%] 2024-08-08T21:00:16.1989420Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 94%] 2024-08-08T21:00:16.1992387Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 94%] 2024-08-08T21:00:16.1995360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 94%] 2024-08-08T21:00:16.1998331Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2001297Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 94%] 2024-08-08T21:00:16.2004264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 94%] 2024-08-08T21:00:16.2007239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 94%] 2024-08-08T21:00:16.2010291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 94%] 2024-08-08T21:00:16.2013293Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 94%] 2024-08-08T21:00:16.2016261Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 94%] 2024-08-08T21:00:16.2019239Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 94%] 2024-08-08T21:00:16.2022225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 94%] 2024-08-08T21:00:16.2025217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 94%] 2024-08-08T21:00:16.2028187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0064s] [ 94%] 2024-08-08T21:00:16.2031197Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 94%] 2024-08-08T21:00:16.2034183Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 94%] 2024-08-08T21:00:16.2037152Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 94%] 2024-08-08T21:00:16.2040116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 94%] 2024-08-08T21:00:16.2043102Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 94%] 2024-08-08T21:00:16.2046083Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 94%] 2024-08-08T21:00:16.2049054Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 94%] 2024-08-08T21:00:16.2051991Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 94%] 2024-08-08T21:00:16.2055000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 94%] 2024-08-08T21:00:16.2058080Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 94%] 2024-08-08T21:00:16.2061475Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 94%] 2024-08-08T21:00:16.2064455Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2067424Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2070391Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 94%] 2024-08-08T21:00:16.2073360Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 94%] 2024-08-08T21:00:16.2076390Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 94%] 2024-08-08T21:00:16.2079427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 94%] 2024-08-08T21:00:16.2082397Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 94%] 2024-08-08T21:00:16.2085304Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 94%] 2024-08-08T21:00:16.2088265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 94%] 2024-08-08T21:00:16.2091225Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 94%] 2024-08-08T21:00:16.2094201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2097155Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_64_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2100174Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 94%] 2024-08-08T21:00:16.2103235Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 94%] 2024-08-08T21:00:16.2106212Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 94%] 2024-08-08T21:00:16.2109192Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 94%] 2024-08-08T21:00:16.2112177Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2115179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2118151Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 94%] 2024-08-08T21:00:16.2121190Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 94%] 2024-08-08T21:00:16.2124229Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 94%] 2024-08-08T21:00:16.2127211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 94%] 2024-08-08T21:00:16.2130164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 94%] 2024-08-08T21:00:16.2133116Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 94%] 2024-08-08T21:00:16.2136094Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 94%] 2024-08-08T21:00:16.2139088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 94%] 2024-08-08T21:00:16.2142068Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2145160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 94%] 2024-08-08T21:00:16.2148193Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0059s] [ 94%] 2024-08-08T21:00:16.2151156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 94%] 2024-08-08T21:00:16.2154120Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 94%] 2024-08-08T21:00:16.2157163Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 94%] 2024-08-08T21:00:16.2160552Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2163540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 94%] 2024-08-08T21:00:16.2166605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 94%] 2024-08-08T21:00:16.2169777Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 94%] 2024-08-08T21:00:16.2172654Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0060s] [ 94%] 2024-08-08T21:00:16.2175625Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 94%] 2024-08-08T21:00:16.2178591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 94%] 2024-08-08T21:00:16.2181570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 94%] 2024-08-08T21:00:16.2184520Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0069s] [ 94%] 2024-08-08T21:00:16.2187496Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 94%] 2024-08-08T21:00:16.2190568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 94%] 2024-08-08T21:00:16.2193610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 94%] 2024-08-08T21:00:16.2196586Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0068s] [ 94%] 2024-08-08T21:00:16.2199568Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 94%] 2024-08-08T21:00:16.2202538Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0060s] [ 94%] 2024-08-08T21:00:16.2205534Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 94%] 2024-08-08T21:00:16.2208511Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0073s] [ 95%] 2024-08-08T21:00:16.2211560Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 95%] 2024-08-08T21:00:16.2214605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 95%] 2024-08-08T21:00:16.2217581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 95%] 2024-08-08T21:00:16.2220596Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 95%] 2024-08-08T21:00:16.2223521Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 95%] 2024-08-08T21:00:16.2226492Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0062s] [ 95%] 2024-08-08T21:00:16.2229464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 95%] 2024-08-08T21:00:16.2232436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 95%] 2024-08-08T21:00:16.2235404Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 95%] 2024-08-08T21:00:16.2238451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 95%] 2024-08-08T21:00:16.2241510Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 95%] 2024-08-08T21:00:16.2244487Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 95%] 2024-08-08T21:00:16.2247464Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 95%] 2024-08-08T21:00:16.2250427Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 95%] 2024-08-08T21:00:16.2253401Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0052s] [ 95%] 2024-08-08T21:00:16.2256432Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 95%] 2024-08-08T21:00:16.2259856Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 95%] 2024-08-08T21:00:16.2263001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 95%] 2024-08-08T21:00:16.2265989Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 95%] 2024-08-08T21:00:16.2268941Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 95%] 2024-08-08T21:00:16.2271913Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 95%] 2024-08-08T21:00:16.2274868Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 95%] 2024-08-08T21:00:16.2277835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 95%] 2024-08-08T21:00:16.2280803Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 95%] 2024-08-08T21:00:16.2283834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 95%] 2024-08-08T21:00:16.2286873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 95%] 2024-08-08T21:00:16.2289835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_128_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 95%] 2024-08-08T21:00:16.2292810Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0072s] [ 95%] 2024-08-08T21:00:16.2295818Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 95%] 2024-08-08T21:00:16.2298797Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 95%] 2024-08-08T21:00:16.2301757Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 95%] 2024-08-08T21:00:16.2304793Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0079s] [ 95%] 2024-08-08T21:00:16.2307844Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 95%] 2024-08-08T21:00:16.2310824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 95%] 2024-08-08T21:00:16.2313813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 95%] 2024-08-08T21:00:16.2316807Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 95%] 2024-08-08T21:00:16.2319783Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 95%] 2024-08-08T21:00:16.2322733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0059s] [ 95%] 2024-08-08T21:00:16.2325710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 95%] 2024-08-08T21:00:16.2328735Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 95%] 2024-08-08T21:00:16.2331769Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0067s] [ 95%] 2024-08-08T21:00:16.2334728Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 95%] 2024-08-08T21:00:16.2337717Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 95%] 2024-08-08T21:00:16.2340700Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0079s] [ 95%] 2024-08-08T21:00:16.2343690Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0080s] [ 95%] 2024-08-08T21:00:16.2346672Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 95%] 2024-08-08T21:00:16.2349704Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 95%] 2024-08-08T21:00:16.2352729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0086s] [ 95%] 2024-08-08T21:00:16.2355759Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0086s] [ 95%] 2024-08-08T21:00:16.2359108Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 95%] 2024-08-08T21:00:16.2362214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 95%] 2024-08-08T21:00:16.2365179Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0063s] [ 95%] 2024-08-08T21:00:16.2368127Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 95%] 2024-08-08T21:00:16.2371095Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 95%] 2024-08-08T21:00:16.2374154Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 95%] 2024-08-08T21:00:16.2377860Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0074s] [ 95%] 2024-08-08T21:00:16.2380847Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0070s] [ 95%] 2024-08-08T21:00:16.2383827Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 95%] 2024-08-08T21:00:16.2386800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 95%] 2024-08-08T21:00:16.2389776Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0098s] [ 95%] 2024-08-08T21:00:16.2392755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0104s] [ 95%] 2024-08-08T21:00:16.2395800Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0066s] [ 95%] 2024-08-08T21:00:16.2398878Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0068s] [ 95%] 2024-08-08T21:00:16.2401852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0105s] [ 95%] 2024-08-08T21:00:16.2404825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0105s] [ 95%] 2024-08-08T21:00:16.2407813Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0075s] [ 95%] 2024-08-08T21:00:16.2410806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 95%] 2024-08-08T21:00:16.2413770Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0074s] [ 95%] 2024-08-08T21:00:16.2416766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0075s] [ 95%] 2024-08-08T21:00:16.2419748Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 95%] 2024-08-08T21:00:16.2422766Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 95%] 2024-08-08T21:00:16.2425725Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0078s] [ 95%] 2024-08-08T21:00:16.2428720Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0078s] [ 95%] 2024-08-08T21:00:16.2431710Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0079s] [ 95%] 2024-08-08T21:00:16.2434670Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0074s] [ 95%] 2024-08-08T21:00:16.2437639Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 95%] 2024-08-08T21:00:16.2440658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0069s] [ 95%] 2024-08-08T21:00:16.2443684Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 96%] 2024-08-08T21:00:16.2446652Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 96%] 2024-08-08T21:00:16.2449615Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0075s] [ 96%] 2024-08-08T21:00:16.2452585Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 96%] 2024-08-08T21:00:16.2455612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 96%] 2024-08-08T21:00:16.2458970Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 96%] 2024-08-08T21:00:16.2461971Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 96%] 2024-08-08T21:00:16.2465033Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0059s] [ 96%] 2024-08-08T21:00:16.2468041Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 96%] 2024-08-08T21:00:16.2471004Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 96%] 2024-08-08T21:00:16.2473980Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 96%] 2024-08-08T21:00:16.2476964Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 96%] 2024-08-08T21:00:16.2479927Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 96%] 2024-08-08T21:00:16.2482900Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_256_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 96%] 2024-08-08T21:00:16.2485916Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 96%] 2024-08-08T21:00:16.2488961Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 96%] 2024-08-08T21:00:16.2491924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 96%] 2024-08-08T21:00:16.2494894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 96%] 2024-08-08T21:00:16.2497873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 96%] 2024-08-08T21:00:16.2500842Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 96%] 2024-08-08T21:00:16.2503808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 96%] 2024-08-08T21:00:16.2506781Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 96%] 2024-08-08T21:00:16.2509788Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 96%] 2024-08-08T21:00:16.2512801Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 96%] 2024-08-08T21:00:16.2515752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 96%] 2024-08-08T21:00:16.2518692Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 96%] 2024-08-08T21:00:16.2521658Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 96%] 2024-08-08T21:00:16.2524626Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 96%] 2024-08-08T21:00:16.2527573Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 96%] 2024-08-08T21:00:16.2530587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0064s] [ 96%] 2024-08-08T21:00:16.2533594Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 96%] 2024-08-08T21:00:16.2536545Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 96%] 2024-08-08T21:00:16.2539505Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 96%] 2024-08-08T21:00:16.2542485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 96%] 2024-08-08T21:00:16.2545456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 96%] 2024-08-08T21:00:16.2548436Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 96%] 2024-08-08T21:00:16.2551381Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 96%] 2024-08-08T21:00:16.2554408Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 96%] 2024-08-08T21:00:16.2557462Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 96%] 2024-08-08T21:00:16.2560792Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 96%] 2024-08-08T21:00:16.2563756Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 96%] 2024-08-08T21:00:16.2566707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 96%] 2024-08-08T21:00:16.2569655Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 96%] 2024-08-08T21:00:16.2572739Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 96%] 2024-08-08T21:00:16.2575808Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 96%] 2024-08-08T21:00:16.2579259Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 96%] 2024-08-08T21:00:16.2582233Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 96%] 2024-08-08T21:00:16.2585194Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 96%] 2024-08-08T21:00:16.2588169Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 96%] 2024-08-08T21:00:16.2591160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 96%] 2024-08-08T21:00:16.2594142Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 96%] 2024-08-08T21:00:16.2597133Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 96%] 2024-08-08T21:00:16.2600202Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 96%] 2024-08-08T21:00:16.2603252Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 96%] 2024-08-08T21:00:16.2606221Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 96%] 2024-08-08T21:00:16.2609175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 96%] 2024-08-08T21:00:16.2612138Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 96%] 2024-08-08T21:00:16.2615124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0058s] [ 96%] 2024-08-08T21:00:16.2618043Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0066s] [ 96%] 2024-08-08T21:00:16.2621056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 96%] 2024-08-08T21:00:16.2624088Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 96%] 2024-08-08T21:00:16.2627092Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 96%] 2024-08-08T21:00:16.2630045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 96%] 2024-08-08T21:00:16.2633019Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 96%] 2024-08-08T21:00:16.2635981Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 96%] 2024-08-08T21:00:16.2638948Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 96%] 2024-08-08T21:00:16.2641917Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 96%] 2024-08-08T21:00:16.2644898Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 96%] 2024-08-08T21:00:16.2647918Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 96%] 2024-08-08T21:00:16.2650922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 96%] 2024-08-08T21:00:16.2653882Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 96%] 2024-08-08T21:00:16.2656885Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 96%] 2024-08-08T21:00:16.2660242Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0057s] [ 96%] 2024-08-08T21:00:16.2663347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 96%] 2024-08-08T21:00:16.2666392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 96%] 2024-08-08T21:00:16.2669451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 96%] 2024-08-08T21:00:16.2672500Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0064s] [ 96%] 2024-08-08T21:00:16.2675563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_4_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 96%] 2024-08-08T21:00:16.2678457Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0112s] [ 97%] 2024-08-08T21:00:16.2681567Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0112s] [ 97%] 2024-08-08T21:00:16.2684460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0068s] [ 97%] 2024-08-08T21:00:16.2687443Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 97%] 2024-08-08T21:00:16.2690550Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0126s] [ 97%] 2024-08-08T21:00:16.2693635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0128s] [ 97%] 2024-08-08T21:00:16.2696619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0083s] [ 97%] 2024-08-08T21:00:16.2699628Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0082s] [ 97%] 2024-08-08T21:00:16.2702702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0075s] [ 97%] 2024-08-08T21:00:16.2705605Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0074s] [ 97%] 2024-08-08T21:00:16.2708590Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 97%] 2024-08-08T21:00:16.2711570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0070s] [ 97%] 2024-08-08T21:00:16.2714614Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0080s] [ 97%] 2024-08-08T21:00:16.2717703Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0081s] [ 97%] 2024-08-08T21:00:16.2720643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0074s] [ 97%] 2024-08-08T21:00:16.2723635Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0075s] [ 97%] 2024-08-08T21:00:16.2726644Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0125s] [ 97%] 2024-08-08T21:00:16.2729666Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0129s] [ 97%] 2024-08-08T21:00:16.2732643Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0078s] [ 97%] 2024-08-08T21:00:16.2735619Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0087s] [ 97%] 2024-08-08T21:00:16.2738681Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0141s] [ 97%] 2024-08-08T21:00:16.2741744Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.1892s] [ 97%] 2024-08-08T21:00:16.2744745Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0096s] [ 97%] 2024-08-08T21:00:16.2747754Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0095s] [ 97%] 2024-08-08T21:00:16.2750758Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0091s] [ 97%] 2024-08-08T21:00:16.2753741Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0091s] [ 97%] 2024-08-08T21:00:16.2756755Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0079s] [ 97%] 2024-08-08T21:00:16.2760205Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0081s] [ 97%] 2024-08-08T21:00:16.2763280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0097s] [ 97%] 2024-08-08T21:00:16.2766279Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0099s] [ 97%] 2024-08-08T21:00:16.2769268Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0087s] [ 97%] 2024-08-08T21:00:16.2772260Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0092s] [ 97%] 2024-08-08T21:00:16.2775271Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0164s] [ 97%] 2024-08-08T21:00:16.2778285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0165s] [ 97%] 2024-08-08T21:00:16.2781264Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0105s] [ 97%] 2024-08-08T21:00:16.2784347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0107s] [ 97%] 2024-08-08T21:00:16.2785942Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0180s] [ 97%] 2024-08-08T21:00:16.2787498Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0181s] [ 97%] 2024-08-08T21:00:16.2789000Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0121s] [ 97%] 2024-08-08T21:00:16.2790543Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0120s] [ 97%] 2024-08-08T21:00:16.2792079Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0107s] [ 97%] 2024-08-08T21:00:16.2793610Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0106s] [ 97%] 2024-08-08T21:00:16.2795191Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0098s] [ 97%] 2024-08-08T21:00:16.2796774Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0099s] [ 97%] 2024-08-08T21:00:16.2798334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0120s] [ 97%] 2024-08-08T21:00:16.2799849Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0118s] [ 97%] 2024-08-08T21:00:16.2801395Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0110s] [ 97%] 2024-08-08T21:00:16.2802931Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0110s] [ 97%] 2024-08-08T21:00:16.2804451Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0105s] [ 97%] 2024-08-08T21:00:16.2806003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0107s] [ 97%] 2024-08-08T21:00:16.2807569Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0070s] [ 97%] 2024-08-08T21:00:16.2809164Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0073s] [ 97%] 2024-08-08T21:00:16.2810683Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0117s] [ 97%] 2024-08-08T21:00:16.2812241Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0116s] [ 97%] 2024-08-08T21:00:16.2813773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0078s] [ 97%] 2024-08-08T21:00:16.2815325Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0078s] [ 97%] 2024-08-08T21:00:16.2816835Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0069s] [ 97%] 2024-08-08T21:00:16.2818418Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0072s] [ 97%] 2024-08-08T21:00:16.2819988Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0067s] [ 97%] 2024-08-08T21:00:16.2821519Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0069s] [ 97%] 2024-08-08T21:00:16.2823048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0077s] [ 97%] 2024-08-08T21:00:16.2824581Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0076s] [ 97%] 2024-08-08T21:00:16.2826123Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0081s] [ 97%] 2024-08-08T21:00:16.2827648Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_512_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0076s] [ 97%] 2024-08-08T21:00:16.2829185Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 97%] 2024-08-08T21:00:16.2830824Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 97%] 2024-08-08T21:00:16.2832355Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 97%] 2024-08-08T21:00:16.2833884Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 97%] 2024-08-08T21:00:16.2835407Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 97%] 2024-08-08T21:00:16.2836969Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 97%] 2024-08-08T21:00:16.2838494Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 97%] 2024-08-08T21:00:16.2840040Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 97%] 2024-08-08T21:00:16.2841593Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 97%] 2024-08-08T21:00:16.2843187Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 97%] 2024-08-08T21:00:16.2844707Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 97%] 2024-08-08T21:00:16.2846243Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0056s] [ 97%] 2024-08-08T21:00:16.2847773Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0067s] [ 97%] 2024-08-08T21:00:16.2849321Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 97%] 2024-08-08T21:00:16.2850837Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 98%] 2024-08-08T21:00:16.2852362Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 98%] 2024-08-08T21:00:16.2853956Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 98%] 2024-08-08T21:00:16.2855583Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 98%] 2024-08-08T21:00:16.2857124Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 98%] 2024-08-08T21:00:16.2859003Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 98%] 2024-08-08T21:00:16.2860600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 98%] 2024-08-08T21:00:16.2862128Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 98%] 2024-08-08T21:00:16.2863665Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0061s] [ 98%] 2024-08-08T21:00:16.2865285Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 98%] 2024-08-08T21:00:16.2866876Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0058s] [ 98%] 2024-08-08T21:00:16.2868419Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 98%] 2024-08-08T21:00:16.2869924Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 98%] 2024-08-08T21:00:16.2871489Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 98%] 2024-08-08T21:00:16.2872992Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 98%] 2024-08-08T21:00:16.2874542Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 98%] 2024-08-08T21:00:16.2876055Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 98%] 2024-08-08T21:00:16.2877662Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 98%] 2024-08-08T21:00:16.2879249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0061s] [ 98%] 2024-08-08T21:00:16.2880812Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0060s] [ 98%] 2024-08-08T21:00:16.2882356Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0058s] [ 98%] 2024-08-08T21:00:16.2883873Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 98%] 2024-08-08T21:00:16.2885460Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 98%] 2024-08-08T21:00:16.2886940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 98%] 2024-08-08T21:00:16.2888570Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0066s] [ 98%] 2024-08-08T21:00:16.2890162Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0065s] [ 98%] 2024-08-08T21:00:16.2891693Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0062s] [ 98%] 2024-08-08T21:00:16.2893217Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 98%] 2024-08-08T21:00:16.2894752Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0061s] [ 98%] 2024-08-08T21:00:16.2896280Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0061s] [ 98%] 2024-08-08T21:00:16.2897802Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0068s] [ 98%] 2024-08-08T21:00:16.2899347Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0068s] [ 98%] 2024-08-08T21:00:16.2900910Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0069s] [ 98%] 2024-08-08T21:00:16.2902508Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0071s] [ 98%] 2024-08-08T21:00:16.2904021Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 98%] 2024-08-08T21:00:16.2905563Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0052s] [ 98%] 2024-08-08T21:00:16.2907081Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 98%] 2024-08-08T21:00:16.2908636Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 98%] 2024-08-08T21:00:16.2910156Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 98%] 2024-08-08T21:00:16.2911729Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 98%] 2024-08-08T21:00:16.2913319Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 98%] 2024-08-08T21:00:16.2914852Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 98%] 2024-08-08T21:00:16.2916374Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 98%] 2024-08-08T21:00:16.2917894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 98%] 2024-08-08T21:00:16.2919450Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 98%] 2024-08-08T21:00:16.2920950Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0057s] [ 98%] 2024-08-08T21:00:16.2922482Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 98%] 2024-08-08T21:00:16.2924006Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 98%] 2024-08-08T21:00:16.2925612Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 98%] 2024-08-08T21:00:16.2927160Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_64_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 98%] 2024-08-08T21:00:16.2928673Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 98%] 2024-08-08T21:00:16.2930249Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 98%] 2024-08-08T21:00:16.2931789Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0053s] [ 98%] 2024-08-08T21:00:16.2933291Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 98%] 2024-08-08T21:00:16.2934865Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0063s] [ 98%] 2024-08-08T21:00:16.2936413Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 98%] 2024-08-08T21:00:16.2938048Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 98%] 2024-08-08T21:00:16.2939547Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 98%] 2024-08-08T21:00:16.2941056Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0056s] [ 98%] 2024-08-08T21:00:16.2942591Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0055s] [ 98%] 2024-08-08T21:00:16.2944115Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 98%] 2024-08-08T21:00:16.2945630Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 98%] 2024-08-08T21:00:16.2947161Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 98%] 2024-08-08T21:00:16.2948733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0065s] [ 98%] 2024-08-08T21:00:16.2950313Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 98%] 2024-08-08T21:00:16.2951834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_16_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 98%] 2024-08-08T21:00:16.2953364Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0054s] [ 98%] 2024-08-08T21:00:16.2954922Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0053s] [ 98%] 2024-08-08T21:00:16.2956485Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 98%] 2024-08-08T21:00:16.2958017Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0051s] [ 98%] 2024-08-08T21:00:16.2960027Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0062s] [ 98%] 2024-08-08T21:00:16.2961678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0061s] [ 98%] 2024-08-08T21:00:16.2963199Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0060s] [ 98%] 2024-08-08T21:00:16.2964763Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0060s] [ 98%] 2024-08-08T21:00:16.2966258Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 98%] 2024-08-08T21:00:16.2967806Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0058s] [ 98%] 2024-08-08T21:00:16.2969308Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0055s] [ 98%] 2024-08-08T21:00:16.2970841Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0055s] [ 98%] 2024-08-08T21:00:16.2972425Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 99%] 2024-08-08T21:00:16.2974013Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0066s] [ 99%] 2024-08-08T21:00:16.2975540Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0067s] [ 99%] 2024-08-08T21:00:16.2977062Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_32_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 99%] 2024-08-08T21:00:16.2978600Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 99%] 2024-08-08T21:00:16.2980137Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 99%] 2024-08-08T21:00:16.2981678Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0052s] [ 99%] 2024-08-08T21:00:16.2983227Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0053s] [ 99%] 2024-08-08T21:00:16.2984822Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 99%] 2024-08-08T21:00:16.2986358Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0062s] [ 99%] 2024-08-08T21:00:16.2987894Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0062s] [ 99%] 2024-08-08T21:00:16.2989428Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0062s] [ 99%] 2024-08-08T21:00:16.2990940Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0057s] [ 99%] 2024-08-08T21:00:16.2992513Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0056s] [ 99%] 2024-08-08T21:00:16.2993985Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0056s] [ 99%] 2024-08-08T21:00:16.2995587Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 99%] 2024-08-08T21:00:16.2997182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0064s] [ 99%] 2024-08-08T21:00:16.2998669Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 99%] 2024-08-08T21:00:16.3000182Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0063s] [ 99%] 2024-08-08T21:00:16.3001733Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_64_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0063s] [ 99%] 2024-08-08T21:00:16.3003244Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0053s] [ 99%] 2024-08-08T21:00:16.3004765Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0054s] [ 99%] 2024-08-08T21:00:16.3006334Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0051s] [ 99%] 2024-08-08T21:00:16.3007901Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0050s] [ 99%] 2024-08-08T21:00:16.3009437Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0061s] [ 99%] 2024-08-08T21:00:16.3010955Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0063s] [ 99%] 2024-08-08T21:00:16.3012490Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0059s] [ 99%] 2024-08-08T21:00:16.3014020Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_False_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0059s] [ 99%] 2024-08-08T21:00:16.3015535Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale0_cuda_float16 PASSED [0.0055s] [ 99%] 2024-08-08T21:00:16.3017045Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float16_scale_l1_cuda_float16 PASSED [0.0057s] [ 99%] 2024-08-08T21:00:16.3018617Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale0_cuda_float32 PASSED [0.0054s] [ 99%] 2024-08-08T21:00:16.3020175Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_0_float32_scale_l1_cuda_float32 PASSED [0.0054s] [ 99%] 2024-08-08T21:00:16.3021682Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale0_cuda_float16 PASSED [0.0065s] [ 99%] 2024-08-08T21:00:16.3023211Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float16_scale_l1_cuda_float16 PASSED [0.0064s] [ 99%] 2024-08-08T21:00:16.3024722Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale0_cuda_float32 PASSED [0.0065s] [ 99%] 2024-08-08T21:00:16.3026265Z test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_vs_math_ref_grads_batch_size_8_seq_len_q_8_seq_len_k_8_head_dim_8_is_causal_True_dropout_p_0_22_float32_scale_l1_cuda_float32 PASSED [0.0066s] [ 99%] 2024-08-08T21:00:16.3027445Z test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_accuracy_type_dense_fused_kernel0_cuda PASSED [0.0284s] [ 99%] 2024-08-08T21:00:16.3028646Z test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_accuracy_type_nested_fused_kernel0_cuda PASSED [0.0511s] [ 99%] 2024-08-08T21:00:16.3029857Z test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_type_dense_is_contiguous_False_cuda PASSED [0.0084s] [ 99%] 2024-08-08T21:00:16.3031059Z test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_type_dense_is_contiguous_True_cuda PASSED [0.0082s] [ 99%] 2024-08-08T21:00:16.3032256Z test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_type_nested_is_contiguous_False_cuda PASSED [0.0349s] [ 99%] 2024-08-08T21:00:16.3033392Z test_transformers.py::TestSDPACudaOnlyCUDA::test_scaled_dot_product_attention_fused_kernels_packed_type_nested_is_contiguous_True_cuda PASSED [0.0331s] [ 99%] 2024-08-08T21:00:16.3034328Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_choice_with_determinism_warn_only_False_cuda PASSED [0.0015s] [ 99%] 2024-08-08T21:00:16.3035238Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_choice_with_determinism_warn_only_True_cuda PASSED [0.0015s] [ 99%] 2024-08-08T21:00:16.3036834Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_False_is_causal_False_bfloat16_cuda_bfloat16 SKIPPED [0.0003s] (Flash Attention was not built for this system) [ 99%] 2024-08-08T21:00:16.3038426Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_False_is_causal_False_float16_cuda_float16 SKIPPED [0.0002s] (Flash Attention was not built for this system) [ 99%] 2024-08-08T21:00:16.3040001Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_False_is_causal_True_bfloat16_cuda_bfloat16 SKIPPED [0.0002s] (Flash Attention was not built for this system) [ 99%] 2024-08-08T21:00:16.3041579Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_False_is_causal_True_float16_cuda_float16 SKIPPED [0.0002s] (Flash Attention was not built for this system) [ 99%] 2024-08-08T21:00:16.3043201Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_True_is_causal_False_bfloat16_cuda_bfloat16 SKIPPED [0.0002s] (Flash Attention was not built for this system) [ 99%] 2024-08-08T21:00:16.3044825Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_True_is_causal_False_float16_cuda_float16 SKIPPED [0.0002s] (Flash Attention was not built for this system) [ 99%] 2024-08-08T21:00:16.3046440Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_True_is_causal_True_bfloat16_cuda_bfloat16 SKIPPED [0.0002s] (Flash Attention was not built for this system) [ 99%] 2024-08-08T21:00:16.3047959Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_flash_attention_grad_against_math_contiguous_inputs_True_is_causal_True_float16_cuda_float16 SKIPPED [0.0002s] (Flash Attention was not built for this system) [ 99%] 2024-08-08T21:00:16.3049110Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_mem_efficient_grad_against_math_contiguous_inputs_False_is_causal_False_cuda PASSED [0.0041s] [ 99%] 2024-08-08T21:00:16.3050214Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_mem_efficient_grad_against_math_contiguous_inputs_False_is_causal_True_cuda PASSED [0.0036s] [ 99%] 2024-08-08T21:00:16.3051353Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_mem_efficient_grad_against_math_contiguous_inputs_True_is_causal_False_cuda PASSED [0.0035s] [ 99%] 2024-08-08T21:00:16.3052456Z test_transformers.py::TestSDPACudaOnlyCUDA::test_sdp_mem_efficient_grad_against_math_contiguous_inputs_True_is_causal_True_cuda PASSED [0.0036s] [ 99%] 2024-08-08T21:00:16.3053702Z test_transformers.py::TestSDPACudaOnlyCUDA::test_singelton_head_dim_stride_ne_1_cuda SKIPPED [0.0002s] (Fused SDPA was not built for this system) [ 99%] 2024-08-08T21:00:16.3054727Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_LOWER_RIGHT_shape0_cuda PASSED [0.0480s] [ 99%] 2024-08-08T21:00:16.3055813Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_LOWER_RIGHT_shape1_cuda PASSED [0.0992s] [ 99%] 2024-08-08T21:00:16.3057516Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_LOWER_RIGHT_shape2_cuda SKIPPED [0.0013s] (Lower right causal mask will produce NaNs in the output when seq_len_q > seq_len_kv!) [ 99%] 2024-08-08T21:00:16.3058819Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_LOWER_RIGHT_shape3_cuda PASSED [0.0042s] [ 99%] 2024-08-08T21:00:16.3059892Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_UPPER_LEFT_shape0_cuda PASSED [0.0480s] [ 99%] 2024-08-08T21:00:16.3060912Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_UPPER_LEFT_shape1_cuda PASSED [0.2458s] [ 99%] 2024-08-08T21:00:16.3061932Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_UPPER_LEFT_shape2_cuda PASSED [0.0980s] [ 99%] 2024-08-08T21:00:16.3062938Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_causal_variant_CausalVariant_UPPER_LEFT_shape3_cuda PASSED [0.0046s] [ 99%] 2024-08-08T21:00:16.3064011Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_LOWER_RIGHT_shape0_cuda PASSED [0.8160s] [ 99%] 2024-08-08T21:00:16.3065098Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_LOWER_RIGHT_shape1_cuda PASSED [0.2246s] [ 99%] 2024-08-08T21:00:16.3066884Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_LOWER_RIGHT_shape2_cuda SKIPPED [0.0034s] (Lower right causal mask will produce NaNs in the output when seq_len_q > seq_len_kv!) [ 99%] 2024-08-08T21:00:16.3068049Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_LOWER_RIGHT_shape3_cuda PASSED [0.1843s] [ 99%] 2024-08-08T21:00:16.3069113Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_UPPER_LEFT_shape0_cuda PASSED [0.1288s] [ 99%] 2024-08-08T21:00:16.3070188Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_UPPER_LEFT_shape1_cuda PASSED [0.1526s] [ 99%] 2024-08-08T21:00:16.3071240Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_UPPER_LEFT_shape2_cuda PASSED [0.1769s] [ 99%] 2024-08-08T21:00:16.3072293Z test_transformers.py::TestAttnBiasCUDA::test_causal_variants_compile_causal_variant_CausalVariant_UPPER_LEFT_shape3_cuda PASSED [0.0893s] [ 99%] 2024-08-08T21:00:16.3073071Z test_transformers.py::TestAttnBiasCUDA::test_is_causal_and_mask_fails_cuda PASSED [0.0021s] [ 99%] 2024-08-08T21:00:16.3073894Z test_transformers.py::TestAttnBiasCUDA::test_is_causal_equals_upper_left_shape0_cuda PASSED [0.0110s] [ 99%] 2024-08-08T21:00:16.3074726Z test_transformers.py::TestAttnBiasCUDA::test_is_causal_equals_upper_left_shape1_cuda PASSED [0.0115s] [ 99%] 2024-08-08T21:00:16.3075544Z test_transformers.py::TestAttnBiasCUDA::test_is_causal_equals_upper_left_shape2_cuda PASSED [0.0218s] [ 99%] 2024-08-08T21:00:16.3076361Z test_transformers.py::TestAttnBiasCUDA::test_is_causal_equals_upper_left_shape3_cuda PASSED [0.0022s] [100%] 2024-08-08T21:00:16.3076445Z 2024-08-08T21:00:16.3077599Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_transformers/test_transformers-6f63f59cbd7291bb.xml - 2024-08-08T21:00:16.3078395Z ======== 7758 passed, 25 skipped, 37561 deselected in 125.23s (0:02:05) ======== 2024-08-08T21:00:16.3080492Z The following tests failed and then succeeded when run in a new process['test/test_transformers.py::TestSDPACudaOnlyCUDA::test_mem_efficient_attention_attn_mask_vs_math_ref_grads_batch_size_1_seq_len_q_128_seq_len_k_128_head_dim_16_is_causal_False_dropout_p_0_22_float32_scale0_cuda_float32'] 2024-08-08T21:00:16.3080514Z 2024-08-08T21:00:16.3081314Z FINISHED PRINTING LOG FILE of test_transformers 1/1 (test/test-reports/test_transformers_1.1_44c310efaea94729_.log) 2024-08-08T21:00:16.3081334Z 2024-08-08T21:00:16.3081852Z Running inductor/test_torchinductor_opinfo 1/18 ... [2024-08-08 20:59:59.223666] 2024-08-08T21:00:16.3083817Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor_opinfo.py', '-m', 'not serial', '--shard-id=1', '--num-shards=18', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 20:59:59.224217] 2024-08-08T21:00:16.3083885Z 2024-08-08T21:00:16.3085185Z inductor/test_torchinductor_opinfo 1/18 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_opinfo_1.18_5259973fbe565482_.log 2024-08-08T21:00:16.3228046Z Running 203 items in this shard: test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_H_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___radd___cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__chunk_cat_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__chunk_cat_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__unsafe_masked_index_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__unsafe_masked_index_put_accumulate_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_acosh_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_addbmm_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_addmv_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_addr_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_amin_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_amin_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_angle_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_any_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_any_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_arange_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_argmax_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_argsort_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_as_strided_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_as_strided_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_asinh_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_atan2_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_atleast_2d_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_bitwise_and_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_bitwise_or_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_block_diag_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_broadcast_tensors_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_broadcast_tensors_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cfloat_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_chalf_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cholesky_inverse_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cholesky_inverse_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clamp_max_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clamp_min_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_corrcoef_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cos_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cosh_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cosh_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_count_nonzero_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_count_nonzero_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cumprod_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cumulative_trapezoid_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_deg2rad_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diag_embed_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diag_embed_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diagonal_copy_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diagonal_copy_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diagonal_scatter_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_dist_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_dstack_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_empty_permuted_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_erf_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_exp2_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_exp_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_expand_as_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_expand_copy_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_expand_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_eye_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_eye_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_fft_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_ihfft_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_rfft2_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_rfft_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_rfft_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fill_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_flip_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_float_power_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fmod_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_gather_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ge_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_grid_sampler_2d_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_i0_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_igammac_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_add_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_copy_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_reduce_amin_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_select_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_int_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isinf_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isinf_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isposinf_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isreal_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_item_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_binary_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_binary_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_binary_return_by_ref_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_unary_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_kthvalue_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_lcm_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ldexp_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_diagonal_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_diagonal_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_eigvalsh_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_lstsq_grad_oriented_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_lu_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_lu_factor_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_qr_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_vander_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log2_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log_normal_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log_softmax_with_dtype_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_logical_and_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_long_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_lt_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_lu_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mT_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_select_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_softmin_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_max_binary_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_max_reduction_no_dim_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_maximum_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_meshgrid_list_of_tensors_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_min_binary_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_min_binary_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_min_reduction_no_dim_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mul_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mvlgamma_mvlgamma_p_3_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nan_to_num_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nanmedian_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nanmedian_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ne_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_empty_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_empty_strided_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_full_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_adaptive_avg_pool3d_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_adaptive_max_pool3d_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_avg_pool3d_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_conv_transpose2d_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_cosine_similarity_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_dropout_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_elu_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_embedding_bag_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_interpolate_nearest_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_interpolate_trilinear_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_kl_div_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_leaky_relu_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_max_unpool2d_grad_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_multilabel_margin_loss_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pairwise_distance_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_prelu_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_relu_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_rms_norm_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_scaled_dot_product_attention_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_softplus_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_softsign_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_threshold_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_norm_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_norm_inf_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_pca_lowrank_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_permute_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polygamma_polygamma_n_2_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polygamma_polygamma_n_3_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_pow_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_put_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_rad2deg_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_remainder_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_reshape_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_reshape_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resize__cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resize_as__cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_round_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_round_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_round_decimals_neg_3_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scalar_tensor_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scatter_reduce_amax_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scatter_reduce_amax_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scatter_reduce_mean_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_searchsorted_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_select_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_signal_windows_kaiser_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_softmax_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_bessel_j1_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_erfcx_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_hermite_polynomial_h_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_i0e_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_scaled_modified_bessel_k0_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_scaled_modified_bessel_k1_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_u_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_zeta_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_zeta_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_split_list_args_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_split_with_sizes_copy_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_stack_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sub_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sum_to_size_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_t_copy_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_take_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_take_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tile_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_topk_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_torch_ops_aten__safe_softmax_default_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tril_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tril_indices_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_trunc_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unflatten_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unsafe_chunk_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unsafe_split_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_var_mean_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_view_as_complex_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_view_copy_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_vstack_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_zeros_like_cuda_int32 2024-08-08T21:00:16.3228726Z 2024-08-08T21:00:16.3229351Z Running inductor/test_torchinductor_opinfo 6/18 ... [2024-08-08 21:00:14.861137] 2024-08-08T21:00:16.3231619Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor_opinfo.py', '-m', 'not serial', '--shard-id=6', '--num-shards=18', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:00:14.861628] 2024-08-08T21:00:26.3991179Z 2024-08-08T21:00:26.3994099Z inductor/test_torchinductor_opinfo 6/18 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_opinfo_6.18_af310ee80f22e4d4_.log 2024-08-08T21:00:26.4140231Z Running 202 items in this shard: test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_T_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rdiv___cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rmod___cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rmul___cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rpow___cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__chunk_cat_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__softmax_backward_data_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__unsafe_masked_index_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_abs_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_acosh_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_addcmul_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_addmm_decomposed_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_aminmax_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_argwhere_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_as_strided_copy_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_asin_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_atleast_3d_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_bfloat16_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_bitwise_not_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_block_diag_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_bool_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_byte_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_byte_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ceil_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_char_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clamp_min_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clone_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_conj_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_constant_pad_nd_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_corrcoef_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cos_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cumulative_trapezoid_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diag_embed_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_div_no_rounding_mode_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_dstack_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_empty_like_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_empty_permuted_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_equal_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_erfinv_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_exp2_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_expand_as_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_expand_as_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_expm1_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_fftn_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_fftshift_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_hfft2_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_hfft_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_ifftn_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_ihfft_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_irfft2_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_irfft2_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fill_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fliplr_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_flipud_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_float_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_floor_divide_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fmin_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_frexp_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_full_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_full_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_full_like_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_gather_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_gather_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_gt_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_half_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_heaviside_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_histc_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_histc_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_hsplit_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_i0_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_i0_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_i0_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_add_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_copy_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_put_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_put_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_int_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isclose_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isclose_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isfinite_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isin_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isin_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isinf_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_2inputs_2outputs_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_4inputs_with_extra_args_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_4inputs_with_extra_args_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_unary_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ldexp_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ldexp_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_det_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_eigvals_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_inv_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_lu_factor_ex_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linspace_tensor_overload_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linspace_tensor_overload_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log1p_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log_softmax_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log_softmax_with_dtype_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_logical_not_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_logit_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_lt_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mH_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mT_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_log_softmax_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_scatter_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_sum_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_var_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_max_reduction_no_dim_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_minimum_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mode_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_movedim_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mul_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mv_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nanmedian_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nansum_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_native_dropout_backward_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_empty_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_ones_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_ones_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_ones_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_adaptive_avg_pool1d_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_bilinear_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_dropout2d_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_feature_alpha_dropout_without_train_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_glu_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_kl_div_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_linear_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_max_unpool2d_grad_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_max_unpool3d_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_multi_head_attention_forward_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_multi_head_attention_forward_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_one_hot_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_circular_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_constant_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_constant_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_reflect_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pairwise_distance_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_relu6_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_relu6_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_softsign_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_triplet_margin_loss_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nonzero_static_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nonzero_static_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_norm_inf_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_normal_in_place_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polygamma_polygamma_n_0_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polygamma_polygamma_n_1_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polygamma_polygamma_n_1_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polygamma_polygamma_n_2_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_pow_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_prod_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_put_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_randint_like_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ravel_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_repeat_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_reshape_as_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resolve_neg_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_roll_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_rot90_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_rot90_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_rsqrt_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_searchsorted_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_select_scatter_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sign_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_signal_windows_general_cosine_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_signal_windows_hamming_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sinh_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sinh_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_slice_scatter_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_softmax_with_dtype_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_bessel_y0_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_bessel_y0_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_chebyshev_polynomial_t_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_hermite_polynomial_h_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_laguerre_polynomial_l_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_log_ndtr_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_modified_bessel_i1_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_modified_bessel_k0_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_ndtr_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_t_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_t_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_u_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_xlog1py_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_split_list_args_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_split_with_sizes_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sqrt_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tan_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tanh_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tile_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_torch_ops_aten__efficient_attention_forward_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_trapezoid_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_trapz_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tril_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tril_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_triu_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unique_cuda_uint64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unsafe_split_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unsafe_split_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_view_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_vstack_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_where_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_zero__cuda_float32 2024-08-08T21:00:26.4282266Z 2024-08-08T21:00:30.4069669Z Running inductor/test_torchinductor_opinfo 11/18 ... [2024-08-08 21:00:30.406426] 2024-08-08T21:00:30.4074995Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor_opinfo.py', '-m', 'not serial', '--shard-id=11', '--num-shards=18', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:00:30.406980] 2024-08-08T21:00:41.9934609Z 2024-08-08T21:00:41.9937784Z inductor/test_torchinductor_opinfo 11/18 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_opinfo_11.18_6141d21fc632f41d_.log 2024-08-08T21:00:42.0106918Z Running 201 items in this shard: test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___getitem___cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rand___cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rmod___cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rmod___cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rpow___cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rpow___cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__segment_reduce_offsets_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__unsafe_masked_index_put_accumulate_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_addcmul_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_addmm_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_alias_copy_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_any_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_any_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_argwhere_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_argwhere_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_as_strided_partial_views_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_as_strided_partial_views_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_as_strided_scatter_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_as_strided_scatter_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_asinh_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_atanh_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_atanh_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_baddbmm_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_bitwise_and_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_block_diag_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_broadcast_to_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cartesian_prod_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_chalf_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_char_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clamp_min_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clone_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_column_stack_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_conj_physical_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_constant_pad_nd_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cos_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_count_nonzero_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cummax_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cummax_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cumprod_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cumprod_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cumulative_trapezoid_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diag_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diagonal_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_digamma_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_div_trunc_rounding_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_empty_like_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_empty_permuted_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_eq_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_erf_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_erfinv_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_exp_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_expand_copy_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_expand_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_eye_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_fft2_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_fft2_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_hfftn_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_ihfft_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_ihfftn_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_irfft_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_irfft_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_irfftn_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_flip_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_flipud_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_float_power_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fmin_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_full_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_full_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_grid_sampler_2d_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_gt_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_half_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_histc_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_add_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_add_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_fill_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isclose_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isnan_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isnan_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isposinf_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_2inputs_2outputs_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_binary_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_binary_return_by_ref_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_cross_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_diagonal_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_householder_product_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_matrix_rank_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log2_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_logit_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_logspace_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mT_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_amax_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_amin_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_cumprod_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_select_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_std_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_median_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_min_reduction_with_dim_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_minimum_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_msort_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nan_to_num_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nansum_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_narrow_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_native_dropout_backward_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_full_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_adaptive_max_pool1d_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_alpha_dropout_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_binary_cross_entropy_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_binary_cross_entropy_with_logits_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_conv_transpose3d_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_dropout2d_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_dropout3d_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_gaussian_nll_loss_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_hardtanh_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_interpolate_area_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_interpolate_linear_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_interpolate_nearest-exact_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_l1_loss_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_max_pool2d_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_max_unpool1d_grad_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_mse_loss_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_multilabel_soft_margin_loss_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_replicate_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pairwise_distance_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_relu_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_softmin_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_tanhshrink_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_triplet_margin_with_distance_loss_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_upsample_bilinear_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_normal_number_mean_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ones_like_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ones_like_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_permute_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polar_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polygamma_polygamma_n_0_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polygamma_polygamma_n_3_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_pow_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_rad2deg_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_randint_like_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_randint_like_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_randn_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_real_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_renorm_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_repeat_interleave_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resize__cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resize__cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resolve_neg_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resolve_neg_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resolve_neg_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_roll_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_rsub_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scatter_reduce_amax_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scatter_reduce_prod_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scatter_reduce_sum_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_short_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sigmoid_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sign_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sign_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_signal_windows_gaussian_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_signal_windows_hamming_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_signal_windows_nuttall_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_slice_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sort_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_bessel_j0_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_bessel_y1_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_chebyshev_polynomial_w_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_erfcx_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_hermite_polynomial_he_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_hermite_polynomial_he_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_legendre_polynomial_p_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_modified_bessel_i0_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_modified_bessel_k0_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_modified_bessel_k1_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_ndtri_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_ndtri_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_polygamma_special_polygamma_n_0_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_zeta_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_split_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_split_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_split_list_args_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_split_with_sizes_copy_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sqrt_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_stack_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_stft_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_take_along_dim_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_take_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tanh_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tanh_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_to_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_topk_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_torch_ops_aten__efficient_attention_forward_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_true_divide_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unique_consecutive_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unique_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unravel_index_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_view_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_view_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_vsplit_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_where_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_zero__cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_zeros_like_cuda_uint8 2024-08-08T21:00:42.0248954Z 2024-08-08T21:00:45.8759009Z Running inductor/test_torchinductor_opinfo 16/18 ... [2024-08-08 21:00:45.875272] 2024-08-08T21:00:45.8762609Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor_opinfo.py', '-m', 'not serial', '--shard-id=16', '--num-shards=18', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:00:45.875783] 2024-08-08T21:00:57.3123818Z 2024-08-08T21:00:57.3126486Z inductor/test_torchinductor_opinfo 16/18 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_opinfo_16.18_36dccc83db37efd3_.log 2024-08-08T21:00:57.3267107Z Running 194 items in this shard: test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___getitem___cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___radd___cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rpow___cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rxor___cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__unsafe_masked_index_put_accumulate_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_acos_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_amax_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_as_strided_partial_views_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_asinh_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_atanh_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_atleast_2d_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_atleast_3d_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_bitwise_xor_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_bmm_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cartesian_prod_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cat_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cat_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cdouble_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ceil_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_char_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_chunk_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_chunk_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clamp_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clamp_max_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clone_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_clone_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_combinations_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_combinations_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_copysign_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cos_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cross_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cummax_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_cumulative_trapezoid_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_deg2rad_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diag_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_diagonal_scatter_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_dist_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_empty_like_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_erfinv_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_expand_as_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_fft2_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_fftshift_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_ifft2_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_ifft_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fft_ihfft2_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fill_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_flatten_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_flatten_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_flip_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fliplr_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_flipud_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_float_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_float_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_floor_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_fmod_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_full_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ge_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_heaviside_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_hsplit_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_igammac_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_copy_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_fill_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_reduce_amin_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_reduce_amin_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_index_reduce_prod_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isclose_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_isreal_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_2inputs_2outputs_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_jiterator_unary_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_le_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_le_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_lerp_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_inv_ex_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_lu_solve_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_solve_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_vander_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_vecdot_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_vecdot_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log2_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_log_normal_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_logical_not_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_logical_xor_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_amin_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_amin_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_cumprod_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_cumsum_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_cumsum_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_fill_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_prod_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_scatter_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_scatter_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_masked_std_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_max_pool2d_with_indices_backward_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_max_reduction_with_dim_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_median_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mode_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_mvlgamma_mvlgamma_p_3_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nanmedian_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nanquantile_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_narrow_copy_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_empty_strided_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_new_zeros_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_alpha_dropout_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_channel_shuffle_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_channel_shuffle_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_conv3d_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_conv_transpose1d_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_conv_transpose2d_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_conv_transpose2d_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_gelu_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_huber_loss_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_interpolate_trilinear_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_layer_norm_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_circular_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_constant_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_constant_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_reflect_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pad_reflect_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pixel_unshuffle_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_pixel_unshuffle_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_relu6_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_rrelu_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_silu_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_softmin_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_softmin_with_dtype_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_softsign_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_triplet_margin_loss_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_nn_functional_triplet_margin_loss_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_outer_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_outer_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_polygamma_polygamma_n_3_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_positive_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_randint_like_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_randn_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_ravel_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_reciprocal_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_remainder_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_repeat_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resize_as__cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_resolve_neg_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_rot90_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_rot90_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scatter_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scatter_reduce_prod_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_scatter_reduce_prod_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_short_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sigmoid_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_signal_windows_cosine_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_signbit_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sin_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sinc_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_slice_scatter_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_softmax_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_bessel_j1_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_bessel_y1_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_chebyshev_polynomial_t_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_chebyshev_polynomial_u_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_legendre_polynomial_p_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_ndtr_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_ndtri_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_scaled_modified_bessel_k1_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_v_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_w_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_special_xlog1py_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_square_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_squeeze_multiple_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_stack_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sub_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sum_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_sum_to_size_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_t_cuda_int64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_take_along_dim_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tanh_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tile_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_to_sparse_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_trace_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_transpose_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_trapezoid_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_trapz_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_triangular_solve_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_tril_indices_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unbind_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unbind_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unflatten_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unflatten_cuda_uint8, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unfold_copy_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unfold_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_unique_cuda_float16, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_vdot_cuda_float32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_vdot_cuda_float64, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_view_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_where_cuda_bool, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_zeros_cuda_int32, test/inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_zeros_cuda_int64 2024-08-08T21:00:57.3403198Z 2024-08-08T21:01:01.1523433Z Running functorch/test_ops 4/5 ... [2024-08-08 21:01:01.151727] 2024-08-08T21:01:01.1528402Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'functorch/test_ops.py', '-m', 'not serial', '--shard-id=4', '--num-shards=5', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:01:01.152240] 2024-08-08T21:17:53.7170357Z 2024-08-08T21:17:53.7172067Z functorch/test_ops 4/5 was successful, full logs can be found in artifacts with path test/test-reports/functorch.test_ops_4.5_a1830a078216e27e_.log 2024-08-08T21:17:53.8561402Z Running 2055 items in this shard: test/functorch/test_ops.py::TestOperatorsCUDA::test_extremal_numerics_l1_loss_cuda, test/functorch/test_ops.py::TestOperatorsCUDA::test_extremal_numerics_mse_loss_cuda, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_NumpyExpMarkDirtyAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_T_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad___getitem___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad___radd___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad___rmod___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_addcmul_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_any_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_as_strided_partial_views_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_asin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_asinh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_atan_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_atleast_3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_block_diag_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_bool_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_broadcast_tensors_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_char_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_combinations_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_diag_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_diag_embed_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_double_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_empty_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_empty_permuted_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_eq_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_erf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_expm1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_fft_hfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_fft_ifft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_fft_irfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_fft_rfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_fft_rfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_fliplr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_frac_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_gather_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_half_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_i0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_index_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_index_put_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_index_reduce_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_isin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_isnan_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_isposinf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_item_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_jiterator_binary_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_cholesky_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_cond_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_eigvals_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_eigvalsh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_inv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_matrix_rank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_multi_dot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_pinv_hermitian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_slogdet_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_linalg_vecdot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_log2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_log_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_log_softmax_with_dtype_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_logdet_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_logical_and_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_logspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_lt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_mT_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_masked_cumprod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_masked_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_masked_median_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_masked_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_masked_select_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_masked_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_matmul_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_max_binary_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_max_reduction_no_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_mm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_mul_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_mv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_native_dropout_backward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_neg_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_new_full_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_avg_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_batch_norm_without_cudnn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_celu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_conv2d_stride_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_conv2d_stride_padding_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_conv2d_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_dropout2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_glu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_instance_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_interpolate_bicubic_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_interpolate_linear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_interpolate_nearest-exact_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_kl_div_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_l1_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_leaky_relu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_margin_ranking_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_max_unpool2d_grad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_mse_loss_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_pad_replicate_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_pad_replicate_negative_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_pixel_unshuffle_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_relu6_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_scaled_dot_product_attention_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_selu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_silu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_softplus_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_threshold_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_nn_functional_unfold_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_norm_fro_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_normal_number_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_outer_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_pinverse_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_polygamma_polygamma_n_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_polygamma_polygamma_n_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_polygamma_polygamma_n_4_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_randint_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_randint_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_real_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_round_decimals_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_scatter_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_signal_windows_cosine_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_signal_windows_hann_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_special_chebyshev_polynomial_u_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_special_erfcx_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_special_i0e_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_special_log_ndtr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_special_modified_bessel_i1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_special_ndtr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_special_scaled_modified_bessel_k1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_split_list_args_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_sqrt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_svd_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_t_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_tanh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_tensor_split_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_to_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_topk_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_trapezoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_trunc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_unique_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_unsafe_chunk_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_unsqueeze_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_var_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_vdot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_view_as_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_view_as_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_view_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_view_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_grad_xlogy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_ForwardHasDefaultArgsAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_SelectGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_T_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp___rdiv___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp___rmul___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp__native_batch_norm_legit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp__unsafe_masked_index_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp__upsample_bilinear2d_aa_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_addmm_decomposed_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_addmv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_angle_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_argmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_as_strided_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_asin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_asinh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_atan2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_baddbmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_bernoulli_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_broadcast_shapes_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_bucketize_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_byte_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_cartesian_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_cauchy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_cdouble_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_cfloat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_cholesky_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_clamp_max_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_conj_physical_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_constant_pad_nd_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_cross_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_diag_embed_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_diagflat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_double_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_einsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_equal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_expand_as_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_fft_fftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_fft_fftshift_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_fft_ifftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_fft_ihfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_fft_rfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_flatten_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_float_power_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_geqrf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_histc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_hstack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_igamma_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_index_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_index_put_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_index_reduce_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_index_select_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_inner_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_isin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_item_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_ldexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_linalg_householder_product_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_linalg_ldl_factor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_linalg_lu_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_linalg_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_linalg_slogdet_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_linalg_solve_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_linalg_tensorinv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_linspace_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_linspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_log2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_log_softmax_with_dtype_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_logical_or_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_logspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_masked_amax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_masked_argmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_masked_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_max_pool2d_with_indices_backward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_maximum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_mvlgamma_mvlgamma_p_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_native_batch_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_native_dropout_backward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_new_zeros_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_adaptive_avg_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_adaptive_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_celu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_channel_shuffle_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_conv2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_conv2d_stride_depthwise_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_conv2d_stride_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_conv2d_stride_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_cosine_embedding_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_embedding_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_fractional_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_hardswish_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_instance_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_l1_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_max_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_max_unpool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_max_unpool2d_grad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_mse_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_mse_loss_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_pad_constant_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_pdist_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_poisson_nll_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_relu6_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_rms_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_soft_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_softplus_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_tanhshrink_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_nn_functional_upsample_nearest_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_normal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_permute_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_polar_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_polygamma_polygamma_n_1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_positive_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_put_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_randint_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_repeat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_rsqrt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_scatter_reduce_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_signal_windows_cosine_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_signal_windows_general_hamming_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_signal_windows_hamming_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_signal_windows_hann_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_sinc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_special_bessel_y1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_special_chebyshev_polynomial_v_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_special_hermite_polynomial_he_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_special_i1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_special_i1e_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_special_modified_bessel_k1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_special_ndtr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_split_list_args_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_square_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_stack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_sub_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_sum_to_size_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_svd_lowrank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_t_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_tile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_torch_ops_aten__safe_softmax_default_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_trapezoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_unfold_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_var_unbiased_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_view_as_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_vsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_vstack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvp_zero__cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpjvpvmap_MulGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpjvpvmap_NumpyExpMarkDirtyAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpjvpvmap_SelectAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpjvpvmap_SortGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_MulGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_ScaleGradGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_SelectGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_T_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp__batch_norm_with_update_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp__native_batch_norm_legit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_addmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_amax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_angle_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_asin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_asinh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_atan2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_bmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_bool_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_broadcast_shapes_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_broadcast_to_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_bucketize_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_cdist_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_char_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_clamp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_column_stack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_conj_physical_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_cumsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_cumulative_trapezoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_deg2rad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_diag_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_diagflat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_diagonal_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_div_no_rounding_mode_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_double_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_equal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_expm1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_exponential_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_fft_ifftshift_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_fft_ihfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_fft_ihfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_fft_irfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_fft_rfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_fft_rfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_fliplr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_float_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_floor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_geqrf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_half_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_heaviside_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_i0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_index_put_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_index_reduce_amax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_isclose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_isin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_isinf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_jiterator_2inputs_2outputs_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_jiterator_4inputs_with_extra_args_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_jiterator_binary_return_by_ref_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_kron_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_ldexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_linalg_cholesky_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_linalg_cholesky_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_linalg_ldl_factor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_linalg_lstsq_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_linalg_matrix_rank_hermitian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_linalg_multi_dot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_linalg_tensorinv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_linalg_vander_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_linspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_log_normal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_long_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_mT_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_masked_cumprod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_masked_cumsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_masked_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_masked_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_masked_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_max_binary_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_max_pool2d_with_indices_backward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_max_reduction_with_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_meshgrid_list_of_tensors_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_multinomial_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nanmean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nanmedian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_new_empty_strided_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_adaptive_avg_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_adaptive_max_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_bilinear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_binary_cross_entropy_with_logits_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_conv2d_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_conv2d_stride_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_cosine_embedding_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_embedding_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_embedding_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_gaussian_nll_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_gelu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_hardsigmoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_hinge_embedding_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_huber_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_interpolate_bilinear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_local_response_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_max_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_max_unpool1d_grad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_max_unpool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_multi_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_multilabel_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_nll_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_pad_reflect_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_relu6_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_rrelu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nn_functional_smooth_l1_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_nonzero_static_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_normal_in_place_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_normal_number_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_pinverse_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_polygamma_polygamma_n_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_polygamma_polygamma_n_4_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_rad2deg_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_renorm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_reshape_as_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_resolve_neg_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_roll_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_rot90_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_round_decimals_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_scatter_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_scatter_reduce_sum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_searchsorted_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_select_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_sinc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_slice_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_slice_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_special_airy_ai_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_special_bessel_y0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_special_chebyshev_polynomial_t_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_special_erfcx_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_special_modified_bessel_i0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_special_scaled_modified_bessel_k1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_split_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_split_with_sizes_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_torch_ops_aten__efficient_attention_forward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_transpose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_trapezoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_triangular_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_trunc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_unique_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_vdot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjp_view_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvjpvmap_ScaleGradGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvmap_MulGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvmap_NumpySortAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvmap_SelectAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_jvpvmapvmap_NumpyCubeAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_tensor_with_scalar_list_cuda, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_T_grad_op_jvp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_contiguous_grad_op_jvp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_expand_grad_op_vjp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_list_return_hsplit_grad_op_jvp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_list_return_hsplit_grad_op_vjp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_list_return_split_grad_op_vjp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_list_return_split_list_args_grad_op_vjp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_list_return_vsplit_grad_op_jvp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_mT_grad_op_jvp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_mT_grad_op_vjp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_resolve_conj_grad_op_vjp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_select_grad_op_vjp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_special_grad_op_vjp_cuda, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_transpose_grad_op_jvp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_unfold_grad_op_jvp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_view_then_inplace_view_as_grad_op_vjp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_H_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_MulGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_NumpyCubeAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_NumpyCubeNotComposableAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_NumpyExpMarkDirtyAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_NumpySortAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_ScaleGradGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_SelectGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp___getitem___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp___rmod___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp___rpow___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp__softmax_backward_data_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_abs_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_addr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_alias_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_allclose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_any_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_argsort_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_as_strided_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_broadcast_to_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_cat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_cdouble_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_cholesky_inverse_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_combinations_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_conj_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_copysign_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_cos_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_cosh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_count_nonzero_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_diagonal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_div_no_rounding_mode_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_einsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_empty_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_empty_strided_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_eq_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_exp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_expand_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_fft_hfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_fft_ifft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_fft_ifftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_fft_irfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_fft_irfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_fft_irfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_float_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_floor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_full_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_histc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_index_reduce_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_index_reduce_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_isclose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_isinf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_isnan_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_isneginf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_isposinf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_isreal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_jiterator_4inputs_with_extra_args_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_le_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_lerp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_linalg_eig_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_linalg_ldl_factor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_linalg_lstsq_grad_oriented_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_linalg_lu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_linalg_pinv_singular_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_linalg_qr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_linalg_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_linspace_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_linspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_log10_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_log_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_log_normal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_logdet_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_logical_not_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_logit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_logspace_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_logsumexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_long_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_lt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_masked_argmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_masked_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_masked_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_masked_std_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_masked_sum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_masked_var_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_matmul_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_min_reduction_with_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nanmedian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nanquantile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_narrow_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_native_dropout_backward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_new_zeros_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nextafter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_adaptive_avg_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_alpha_dropout_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_avg_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_avg_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_avg_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_bilinear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_celu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_channel_shuffle_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_conv2d_stride_depthwise_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_conv2d_stride_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_conv2d_stride_padding_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_conv2d_stride_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_conv2d_strided_padding_dilation_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_conv3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_conv_transpose3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_cosine_embedding_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_feature_alpha_dropout_without_train_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_hinge_embedding_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_huber_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_interpolate_linear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_interpolate_nearest-exact_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_multilabel_soft_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_nll_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_pad_circular_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_pad_reflect_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_pad_replicate_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_scaled_dot_product_attention_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_softshrink_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nn_functional_softsign_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_nonzero_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_norm_inf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_outer_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_permute_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_polygamma_polygamma_n_4_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_quantile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_rand_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_randint_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_randn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_remainder_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_resolve_neg_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_rot90_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_rsqrt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_scatter_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_select_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_sgn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_sign_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_signal_windows_blackman_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_signal_windows_hamming_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_signal_windows_hann_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_sinc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_sinh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_slice_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_sparse_sampled_addmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_special_i1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_special_i1e_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_special_laguerre_polynomial_l_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_special_modified_bessel_k1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_special_ndtr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_special_shifted_chebyshev_polynomial_u_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_special_spherical_bessel_j0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_std_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_sub_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_sum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_svd_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_take_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_tanh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_topk_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_true_divide_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_uniform_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_unique_consecutive_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_var_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_where_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjp_xlogy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_ForwardHasDefaultArgsAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_MulGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_NumpyCubeAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_NumpySortAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_T_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp___rdiv___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp___rmatmul___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp___rsub___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp__segment_reduce_offsets_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp__upsample_bilinear2d_aa_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_acosh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_all_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_argwhere_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_as_strided_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_atanh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_atleast_3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_bernoulli_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_bool_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_bucketize_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_ceil_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_cfloat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_char_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_clamp_max_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_clone_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_combinations_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_conj_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_contiguous_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_cummax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_cumsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_deg2rad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_diagflat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_double_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_double_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_empty_strided_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_erfinv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_fft_fft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_fft_fftshift_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_fft_ihfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_fft_ihfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_fft_irfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_fft_rfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_flatten_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_full_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_geometric_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_geqrf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_grid_sampler_2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_gt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_half_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_half_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_index_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_int_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_isin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_item_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_jiterator_4inputs_with_extra_args_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_jiterator_binary_return_by_ref_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_cholesky_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_det_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_det_singular_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_diagonal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_eigh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_householder_product_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_inv_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_ldl_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_lu_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_matrix_power_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_norm_subgradients_at_zero_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_svd_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linalg_vander_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_linspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_log_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_log_normal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_logcumsumexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_logical_and_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_logspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_masked_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_masked_fill_functorch_Scalar_only_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_masked_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_masked_logsumexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_masked_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_masked_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_masked_select_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_masked_softmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_matrix_exp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_max_reduction_no_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_meshgrid_list_of_tensors_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_min_reduction_with_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_mm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_mul_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nan_to_num_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nanmean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_native_dropout_backward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_adaptive_max_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_adaptive_max_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_avg_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_batch_norm_without_cudnn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_binary_cross_entropy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_binary_cross_entropy_with_logits_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_celu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_conv2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_conv2d_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_conv3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_conv_transpose1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_embedding_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_embedding_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_glu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_group_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_huber_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_interpolate_area_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_interpolate_nearest-exact_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_max_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_mse_loss_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_multi_head_attention_forward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_pad_replicate_negative_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_rrelu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_silu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_smooth_l1_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_softshrink_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_nn_functional_upsample_nearest_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_ormqr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_permute_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_polar_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_polygamma_polygamma_n_1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_positive_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_quantile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_rad2deg_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_randn_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_remainder_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_repeat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_reshape_as_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_resize__cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_round_decimals_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_scalar_tensor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_scatter_reduce_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_scatter_reduce_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_scatter_reduce_sum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_signal_windows_nuttall_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_slice_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_special_bessel_j0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_special_bessel_j1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_special_chebyshev_polynomial_u_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_special_log_ndtr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_special_modified_bessel_k0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_special_shifted_chebyshev_polynomial_t_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_special_shifted_chebyshev_polynomial_u_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_split_list_args_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_split_with_sizes_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_split_with_sizes_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_squeeze_multiple_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_std_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_sub_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_sum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_tan_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_tensor_split_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_transpose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_trapezoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_tril_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_true_divide_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_unfold_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjp_where_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjpvmap_NumpySortAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvjpvmap_SortGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_SelectAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_SortGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_ZeroGradientsGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap___rmatmul___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap__segment_reduce_lengths_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_atleast_3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_bmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_bool_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_cartesian_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_cauchy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_clamp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_constant_pad_nd_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_copysign_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_cross_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_cumsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_deg2rad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_diagonal_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_diagonal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_dist_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_div_trunc_rounding_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_double_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_empty_permuted_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_expand_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_fft_rfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_flatten_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_float_power_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_geqrf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_heaviside_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_hsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_i0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_index_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_index_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_index_reduce_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_index_reduce_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_index_select_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_int_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_isclose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_isinf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_jiterator_2inputs_2outputs_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_jiterator_binary_return_by_ref_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_jiterator_unary_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_le_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_lgamma_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_cross_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_diagonal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_eigvalsh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_ldl_factor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_ldl_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_lu_factor_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_matrix_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_matrix_power_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_matrix_rank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_norm_subgradients_at_zero_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_pinv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_tensorinv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_vander_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_linalg_vector_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_log10_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_log_normal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_logaddexp2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_logical_not_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_logical_xor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_logspace_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_masked_amax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_masked_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_masked_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_masked_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_masked_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_matrix_exp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_max_reduction_with_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_min_reduction_no_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_movedim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_mul_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_mvlgamma_mvlgamma_p_1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nansum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_narrow_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_new_ones_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_adaptive_avg_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_adaptive_max_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_adaptive_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_avg_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_ctc_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_glu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_grid_sample_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_hardsigmoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_hinge_embedding_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_max_unpool3d_grad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_nll_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_rrelu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_selu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_nn_functional_unfold_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_normal_in_place_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_normal_number_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_ones_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_ormqr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_polygamma_polygamma_n_2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_polygamma_polygamma_n_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_quantile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_rand_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_randn_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_rot90_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_round_decimals_neg_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_rsub_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_scalar_tensor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_scatter_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_select_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_short_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_sign_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_signal_windows_gaussian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_signal_windows_nuttall_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_slice_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_airy_ai_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_bessel_j0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_bessel_y0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_chebyshev_polynomial_t_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_entr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_laguerre_polynomial_l_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_scaled_modified_bessel_k0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_scaled_modified_bessel_k1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_shifted_chebyshev_polynomial_w_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_spherical_bessel_j0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_xlog1py_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_special_zeta_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_sqrt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_squeeze_multiple_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_stack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_std_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_to_sparse_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_tril_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_trunc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_unflatten_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_unsqueeze_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_unsqueeze_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_var_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_var_mean_unbiased_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_view_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vjpvmap_vstack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_ForwardHasDefaultArgsAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_MulGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_NumpyCubeAutogradFunction_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_NumpyCubeNotComposableAutogradFunction_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_NumpySortAutogradFunction_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_ScaleGradGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_SelectAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_SelectGenVmapAutogradFunction_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_T_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad___getitem___functorch_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad___rmul___cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad___rsub___cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad__batch_norm_with_update_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad__upsample_bilinear2d_aa_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_addbmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_addcdiv_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_addmm_decomposed_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_alias_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_all_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_amin_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_aminmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_arange_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_argmin_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_as_strided_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_as_strided_partial_views_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_as_strided_scatter_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_asin_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_atan_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_atleast_3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_bfloat16_functorch_no_channels_last_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_bool_functorch_no_channels_last_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_broadcast_tensors_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_byte_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_cat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_cat_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_ceil_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_char_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_cholesky_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_clamp_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_clone_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_combinations_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_conj_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_conj_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_conj_physical_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_conj_physical_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_copysign_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_copysign_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_corrcoef_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_cos_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_cov_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_cummin_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_cumprod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_diag_embed_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_diagonal_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_diagonal_scatter_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_diff_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_digamma_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_div_no_rounding_mode_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_dot_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_empty_permuted_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_equal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_expand_as_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_expm1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fft_fft2_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fft_fftn_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fft_ifft2_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fft_ifftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fft_ifftn_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fft_ifftshift_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fft_irfftn_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fliplr_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_float_functorch_no_channels_last_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_fmod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_frac_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_frexp_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_ge_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_geometric_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_gradient_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_hsplit_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_i0_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_igamma_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_igammac_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_index_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_index_reduce_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_index_reduce_amin_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_index_reduce_mean_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_inner_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_inner_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_int_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_isfinite_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_isinf_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_isposinf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_isposinf_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_isreal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_ldexp_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_lerp_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_lgamma_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_cond_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_eig_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_eig_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_inv_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_ldl_factor_ex_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_lstsq_grad_oriented_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_lu_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_matrix_norm_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_matrix_power_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_matrix_power_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_slogdet_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_svd_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_svdvals_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_tensorinv_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linalg_tensorsolve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_linspace_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_log_normal_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_log_softmax_with_dtype_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_logdet_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_logical_not_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_logspace_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_long_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_long_functorch_no_channels_last_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_lt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_lu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_lu_unpack_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_mT_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_masked_amin_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_masked_argmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_masked_argmax_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_masked_argmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_masked_argmin_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_masked_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_masked_fill_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_masked_logsumexp_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_masked_softmin_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_matmul_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_matrix_exp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_minimum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_mvlgamma_mvlgamma_p_1_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_mvlgamma_mvlgamma_p_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_mvlgamma_mvlgamma_p_3_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nan_to_num_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nanmedian_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_narrow_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_native_layer_norm_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_ne_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_new_ones_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_adaptive_avg_pool1d_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_adaptive_avg_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_adaptive_max_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_adaptive_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_avg_pool1d_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_batch_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_batch_norm_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_batch_norm_without_cudnn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_binary_cross_entropy_with_logits_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_conv2d_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_conv2d_stride_no_bias_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_conv2d_stride_padding_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_conv2d_strided_padding_dilation_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_conv2d_strided_padding_dilation_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_cosine_embedding_loss_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_ctc_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_dropout3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_dropout_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_dropout_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_embedding_bag_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_embedding_bag_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_feature_alpha_dropout_without_train_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_glu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_hardswish_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_interpolate_linear_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_interpolate_nearest_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_interpolate_nearest_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_l1_loss_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_layer_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_leaky_relu_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_linear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_local_response_norm_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_logsigmoid_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_max_pool1d_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_max_unpool1d_grad_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_max_unpool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_mish_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_mse_loss_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_mse_loss_functorch_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_multilabel_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_multilabel_soft_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_nll_loss_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_pad_constant_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_pad_constant_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_pad_reflect_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_pad_replicate_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_pixel_shuffle_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_rms_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_silu_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_soft_margin_loss_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_softshrink_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_tanhshrink_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_triplet_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_triplet_margin_with_distance_loss_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nn_functional_unfold_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_nonzero_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_norm_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_normal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_normal_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_normal_number_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_ormqr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_permute_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_polygamma_polygamma_n_2_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_positive_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_qr_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_quantile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_quantile_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_rand_like_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_randn_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_randn_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_real_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_repeat_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_resolve_conj_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_round_decimals_neg_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_scatter_reduce_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_sgn_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_short_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_sigmoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_signal_windows_bartlett_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_signal_windows_blackman_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_signal_windows_gaussian_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_signal_windows_general_cosine_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_signal_windows_hamming_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_signal_windows_kaiser_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_signbit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_sin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_sinc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_sinc_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_slice_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_softmax_with_dtype_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_bessel_j1_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_bessel_y1_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_chebyshev_polynomial_t_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_erfcx_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_i1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_i1_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_i1e_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_legendre_polynomial_p_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_modified_bessel_i1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_modified_bessel_k0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_modified_bessel_k1_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_scaled_modified_bessel_k1_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_shifted_chebyshev_polynomial_t_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_shifted_chebyshev_polynomial_t_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_shifted_chebyshev_polynomial_u_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_shifted_chebyshev_polynomial_v_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_special_xlog1py_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_split_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_split_list_args_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_split_with_sizes_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_square_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_std_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_std_mean_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_std_mean_unbiased_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_stft_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_sub_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_sum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_sum_to_size_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_t_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_tan_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_tensor_split_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_tensordot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_tensordot_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_transpose_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_triangular_solve_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_tril_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_unbind_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_unfold_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_unfold_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_uniform_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_unsafe_chunk_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_unsafe_chunk_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_unsqueeze_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_var_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_var_mean_unbiased_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_vdot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_view_as_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_vsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_vstack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_where_cuda_float64, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmap_autograd_grad_zero__cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_NumpyCubeNotComposableAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_NumpyMulAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_NumpySortAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_ZeroGradientsGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall___rmatmul___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall___rmul___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall__upsample_bilinear2d_aa_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_addbmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_addmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_alias_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_allclose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_angle_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_atleast_1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_bool_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_broadcast_shapes_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_bucketize_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_cdist_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_cdouble_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_ceil_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_contiguous_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_corrcoef_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_cross_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_cummax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_cummin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_cumprod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_diagonal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_diff_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_div_floor_rounding_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_expand_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_eye_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_fft_hfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_fft_ifftshift_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_fft_irfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_fft_irfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_fft_rfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_frac_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_H_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_NumpyCubeNotComposableAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_NumpyMulAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_ScaleGradGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_T_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule___rmatmul___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule__chunk_cat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule__segment_reduce_offsets_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule__softmax_backward_data_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_addcdiv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_addmv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_amax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_aminmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_asin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_atleast_1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_bmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_bool_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_broadcast_tensors_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_byte_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_cartesian_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_cdist_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_clone_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_conj_physical_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_cosh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_cross_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_cumulative_trapezoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_div_trunc_rounding_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_exp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_fft_ifft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_fft_ifftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_fft_rfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_fft_rfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_flip_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_floor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_frac_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_ge_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_histc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_hsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_hypot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_index_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_int_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_isclose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_isfinite_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_isnan_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_isneginf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_jiterator_binary_return_by_ref_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_le_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_lgamma_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_linalg_diagonal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_linalg_ldl_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_linalg_matrix_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_linalg_solve_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_linalg_svdvals_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_linalg_vector_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_log1p_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_log_normal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_log_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_logaddexp2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_logsumexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_mH_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_masked_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_max_binary_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_max_reduction_with_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_maximum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_median_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nanmean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_narrow_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_native_layer_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_adaptive_avg_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_adaptive_avg_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_adaptive_max_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_adaptive_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_avg_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_conv2d_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_conv2d_stride_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_conv2d_strided_padding_dilation_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_cosine_similarity_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_fractional_max_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_gaussian_nll_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_glu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_hardshrink_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_huber_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_interpolate_bilinear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_max_unpool2d_grad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_mse_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_normalize_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_pad_circular_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_pairwise_distance_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_pdist_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_prelu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nn_functional_triplet_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_nonzero_static_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_normal_in_place_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_ones_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_polar_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_polygamma_polygamma_n_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_polygamma_polygamma_n_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_randint_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_randn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_reshape_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_rot90_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_scatter_reduce_sum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_sgn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_signal_windows_bartlett_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_signal_windows_exponential_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_signal_windows_nuttall_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_signbit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_sin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_chebyshev_polynomial_w_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_i0e_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_i1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_i1e_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_modified_bessel_i1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_ndtr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_polygamma_special_polygamma_n_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_shifted_chebyshev_polynomial_v_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_shifted_chebyshev_polynomial_w_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_special_xlog1py_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_sqrt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_squeeze_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_squeeze_multiple_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_stack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_std_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_std_mean_unbiased_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_stft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_take_along_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_torch_ops_aten__safe_softmax_default_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_unflatten_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_unfold_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_var_mean_unbiased_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_vdot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_view_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_has_batch_rule_xlogy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_histc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_hsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_i0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_igammac_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_inner_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_isfinite_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_item_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_jiterator_binary_return_by_ref_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_lerp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_linalg_cholesky_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_linalg_eigvalsh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_linalg_lu_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_linalg_matrix_power_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_linalg_qr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_linalg_svd_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_linalg_vector_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_linspace_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_log1p_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_logspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_lt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_lu_unpack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_masked_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_masked_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_masked_normalize_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_masked_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_masked_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_matmul_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_matrix_exp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_max_pool2d_with_indices_backward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_mm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_narrow_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_ne_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_neg_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_adaptive_max_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_adaptive_max_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_adaptive_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_celu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_conv2d_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_conv2d_stride_padding_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_cosine_similarity_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_ctc_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_embedding_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_feature_alpha_dropout_without_train_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_grid_sample_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_hardswish_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_interpolate_linear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_max_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_max_unpool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_multi_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_multilabel_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_softplus_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_softshrink_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_nn_functional_upsample_bilinear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_normal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_normal_in_place_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_ops_aten__new_zeros_with_same_feature_meta_functorchonly_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_pca_lowrank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_polygamma_polygamma_n_1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_polygamma_polygamma_n_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_remainder_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_resize_as__cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_round_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_scatter_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_scatter_reduce_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_sgn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_short_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_signal_windows_gaussian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_signal_windows_hann_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_sin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_softmax_with_dtype_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_airy_ai_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_chebyshev_polynomial_u_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_entr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_erfcx_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_hermite_polynomial_h_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_hermite_polynomial_he_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_modified_bessel_i0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_ndtr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_ndtri_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_special_polygamma_special_polygamma_n_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_split_with_sizes_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_std_unbiased_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_stft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_svd_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_tensordot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_to_sparse_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_trapz_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_unfold_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_var_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_var_mean_unbiased_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_view_as_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_view_as_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_vstack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_where_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpall_zero__cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_NumpyTakeAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp___rmod___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp__chunk_cat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_argsort_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_as_strided_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_as_strided_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_broadcast_shapes_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_cartesian_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_ceil_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_chalf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_cholesky_inverse_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_combinations_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_conj_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_copysign_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_cummin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_deg2rad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_diagflat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_dist_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_dsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_eq_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_fft_hfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_fft_ifft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_fft_ihfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_fft_ihfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_fft_irfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_float_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_fmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_half_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_heaviside_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_hsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_igammac_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_index_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_isneginf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_jiterator_2inputs_2outputs_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_jiterator_binary_return_by_ref_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_ldexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_le_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_linalg_cross_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_linalg_ldl_factor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_linalg_lstsq_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_linalg_lstsq_grad_oriented_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_linalg_lu_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_linalg_pinv_singular_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_linalg_qr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_linalg_vander_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_linspace_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_log1p_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_log_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_logcumsumexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_logical_and_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_logit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_logsumexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_masked_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_masked_argmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_masked_cumsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_masked_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_masked_normalize_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_masked_std_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_max_reduction_no_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_median_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_mul_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nanmedian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_new_empty_strided_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_new_zeros_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_avg_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_conv2d_stride_depthwise_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_conv2d_stride_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_conv2d_stride_padding_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_embedding_bag_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_glu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_hardshrink_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_hardsigmoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_hardswish_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_interpolate_linear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_kl_div_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_l1_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_layer_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_leaky_relu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_logsigmoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_multi_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_multilabel_soft_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_normalize_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_pad_replicate_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_relu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_nn_functional_unfold_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_ormqr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_put_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_rot90_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_round_decimals_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_scalar_tensor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_scatter_reduce_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_select_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_sigmoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_signal_windows_blackman_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_signal_windows_gaussian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_signal_windows_nuttall_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_sin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_softmax_with_dtype_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_sparse_mm_reduce_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_bessel_j1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_bessel_y1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_chebyshev_polynomial_t_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_chebyshev_polynomial_u_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_erfcx_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_hermite_polynomial_h_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_hermite_polynomial_he_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_legendre_polynomial_p_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_ndtr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_shifted_chebyshev_polynomial_u_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_special_shifted_chebyshev_polynomial_w_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_square_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_squeeze_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_squeeze_multiple_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_std_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_svd_lowrank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_t_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_take_along_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_tan_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_trapz_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_uniform_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvjp_where_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvmap_NumpyCubeAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvmap_NumpyExpMarkDirtyAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvmap_SelectAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvmap_SelectGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapjvpvmap_ZeroGradientsGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_CubeGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_NumpyCubeAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_NumpyCubeNotComposableAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_NumpyMulAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_ZeroGradientsGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp___getitem___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp__batch_norm_with_update_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_acosh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_all_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_angle_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_any_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_argsort_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_bfloat16_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_block_diag_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_broadcast_shapes_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_byte_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_cauchy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_cdouble_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_ceil_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_cfloat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_chalf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_copysign_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_cummax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_diagflat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_div_no_rounding_mode_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_empty_permuted_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_empty_strided_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_equal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_erf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_erfinv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_expand_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_expand_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_fft_hfft2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_fft_hfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_fft_ihfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_fft_rfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_fliplr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_flipud_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_gather_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_geqrf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_NumpyExpMarkDirtyAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_ScaleGradGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_ZeroGradientsGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule__chunk_cat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule__native_batch_norm_legit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule__upsample_bilinear2d_aa_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_acosh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_addmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_alias_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_atleast_3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_bmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_broadcast_tensors_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_ceil_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_chalf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_clone_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_combinations_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_contiguous_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_cross_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_cummin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_cumsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_deg2rad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_diag_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_diagonal_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_double_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_double_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_dsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_dstack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_empty_strided_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_exp2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_expand_as_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_expand_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_eye_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_fft_ifft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_fft_irfftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_fft_rfft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_fliplr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_fmod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_frac_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_geometric_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_hsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_igamma_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_index_put_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_index_select_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_isreal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_kron_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_ldexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_linalg_cholesky_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_linalg_eigvalsh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_linalg_inv_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_linalg_lstsq_grad_oriented_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_linalg_lu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_linalg_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_linalg_vander_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_linspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_log1p_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_log2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_logaddexp2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_logical_xor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_mH_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_masked_cumsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_masked_fill_functorch_Scalar_only_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_masked_median_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_masked_prod_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_masked_softmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_mvlgamma_mvlgamma_p_1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nanmedian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_native_batch_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_native_dropout_backward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_neg_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_new_zeros_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_adaptive_avg_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_adaptive_avg_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_adaptive_max_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_conv2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_conv2d_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_conv2d_stride_padding_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_conv_transpose2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_cosine_embedding_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_ctc_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_dropout2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_dropout3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_embedding_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_feature_alpha_dropout_without_train_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_grid_sample_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_huber_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_interpolate_bicubic_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_max_unpool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_mse_loss_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_nll_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_pad_circular_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_pad_replicate_negative_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_relu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_scaled_dot_product_attention_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_softmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_unfold_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_nn_functional_upsample_nearest_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_normal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_normal_in_place_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_ops_aten__new_zeros_with_same_feature_meta_functorchonly_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_ops_aten_index_put_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_polygamma_polygamma_n_4_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_qr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_quantile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_rad2deg_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_real_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_repeat_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_resolve_conj_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_rot90_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_round_decimals_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_scatter_reduce_amax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_short_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_short_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_signal_windows_bartlett_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_signal_windows_hamming_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_signal_windows_kaiser_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_sin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_sinh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_sort_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_sparse_sampled_addmm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_special_airy_ai_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_special_entr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_special_polygamma_special_polygamma_n_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_special_spherical_bessel_j0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_split_with_sizes_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_squeeze_multiple_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_std_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_stft_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_sub_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_take_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_tile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_to_sparse_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_transpose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_triu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_true_divide_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_trunc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_unbind_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_unfold_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_uniform_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_var_mean_unbiased_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_has_batch_rule_zeros_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_heaviside_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_index_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_index_fill_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_isclose_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_item_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_jiterator_binary_return_by_ref_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_kron_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_ldexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_lerp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_lgamma_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_cond_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_cross_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_det_singular_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_diagonal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_lu_factor_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_matrix_rank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_qr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_slogdet_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_svdvals_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_linalg_vector_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_log1p_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_logaddexp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_logdet_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_long_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_masked_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_masked_cumsum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_masked_fill_functorch_Scalar_only_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_masked_median_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_masked_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_masked_select_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_masked_softmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_min_reduction_no_dim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_minimum_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_mm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_movedim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_multinomial_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_new_empty_strided_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_bilinear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_conv1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_cosine_embedding_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_dropout2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_elu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_huber_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_interpolate_area_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_interpolate_nearest-exact_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_interpolate_nearest_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_interpolate_trilinear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_l1_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_layer_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_leaky_relu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_max_unpool1d_grad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_mse_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_mse_loss_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_pad_circular_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_pad_constant_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_pad_reflect_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_pad_replicate_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_pad_replicate_negative_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_pixel_shuffle_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_relu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_rrelu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_softmin_with_dtype_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_nn_functional_unfold_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_normal_number_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_outer_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_pca_lowrank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_polar_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_polygamma_polygamma_n_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_polygamma_polygamma_n_2_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_positive_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_pow_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_randn_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_renorm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_resolve_conj_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_roll_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_rsqrt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_rsub_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_searchsorted_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_short_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_short_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_signal_windows_nuttall_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_sort_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_special_hermite_polynomial_he_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_special_laguerre_polynomial_l_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_special_modified_bessel_i1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_special_modified_bessel_k1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_special_polygamma_special_polygamma_n_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_split_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_squeeze_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_std_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_sum_to_size_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_svd_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_svd_lowrank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_tensordot_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_trapz_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_unique_consecutive_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_view_as_complex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_view_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_view_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_vsplit_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjp_vstack_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_ForwardHasDefaultArgsAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_NumpyMulAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp___rmatmul___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp___rpow___cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp__softmax_backward_data_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_addmm_decomposed_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_atanh_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_bfloat16_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_bool_functorch_no_channels_last_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_cdouble_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_ceil_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_cholesky_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_conj_physical_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_empty_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_empty_permuted_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_equal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_erf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_exp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_expand_as_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_fft_fftn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_fft_fftshift_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_flip_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_floor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_fmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_full_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_geqrf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_gt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_igamma_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_index_add_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_index_reduce_amin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_isneginf_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_jiterator_2inputs_2outputs_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_jiterator_binary_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_kthvalue_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_lerp_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_linalg_cholesky_ex_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_linalg_diagonal_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_linalg_householder_product_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_linalg_ldl_factor_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_linalg_matrix_rank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_linalg_matrix_rank_hermitian_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_linalg_qr_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_linalg_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_linalg_tensorinv_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_log_softmax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_logspace_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_logspace_tensor_overload_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_lu_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_masked_softmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_masked_std_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_masked_var_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_matmul_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_meshgrid_variadic_tensors_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_min_binary_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_movedim_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_msort_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nanmean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nanquantile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_narrow_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_native_dropout_backward_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_adaptive_avg_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_adaptive_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_conv2d_stride_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_conv2d_stride_padding_no_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_conv2d_stride_with_bias_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_embedding_bag_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_embedding_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_fractional_max_pool2d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_fractional_max_pool3d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_interpolate_bilinear_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_kl_div_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_l1_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_leaky_relu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_local_response_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_logsigmoid_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_max_pool1d_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_max_unpool3d_grad_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_multi_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_pad_replicate_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_relu6_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_rms_norm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_selu_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_soft_margin_loss_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_softmin_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_softshrink_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_nn_functional_threshold_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_normal_number_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_ones_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_ops_aten_index_put_functorch_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_pca_lowrank_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_polygamma_polygamma_n_0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_positive_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_ravel_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_renorm_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_reshape_as_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_round_decimals_neg_3_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_rsqrt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_scatter_reduce_amax_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_select_scatter_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_sgn_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_signal_windows_hann_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_sinc_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_special_chebyshev_polynomial_w_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_special_hermite_polynomial_h_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_special_legendre_polynomial_p_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_special_modified_bessel_k0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_special_modified_bessel_k1_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_special_scaled_modified_bessel_k0_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_sqrt_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_square_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_std_mean_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_std_mean_unbiased_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_std_unbiased_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_tile_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_to_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_triangular_solve_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_tril_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_unfold_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_unsqueeze_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_unsqueeze_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_view_copy_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_zero__cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvjp_zeros_like_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvmap_NumpyCubeNotComposableAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvmap_NumpyExpMarkDirtyAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvmap_NumpyMulAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvjpvmap_ScaleGradGenVmapAutogradFunction_cuda_float32, test/functorch/test_ops.py::TestOperatorsCUDA::test_vmapvmapjvp_linalg_solve_cuda 2024-08-08T21:17:53.9741859Z 2024-08-08T21:17:57.5054698Z Running dynamo/test_dynamic_shapes 1/2 ... [2024-08-08 21:17:57.504859] 2024-08-08T21:17:57.5059500Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'dynamo/test_dynamic_shapes.py', '-m', 'not serial', '--shard-id=1', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:17:57.505409] 2024-08-08T21:23:05.1077301Z 2024-08-08T21:23:05.1079454Z dynamo/test_dynamic_shapes 1/2 was successful, full logs can be found in artifacts with path test/test-reports/dynamo.test_dynamic_shapes_1.2_ef2ced4b5d4bbf1c_.log 2024-08-08T21:23:05.1643118Z Running 812 items in this shard: test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_arguments_binding_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_cpu_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_cpu_graph_break_2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_cpu_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_cpu_graph_break_inner_fn_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_decorator_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_device_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autograd_profiler_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autograd_profiler_enabled_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_context_wrapping_grad_mode_nested_function_decorator_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_cuda_amp_autocast_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_cuda_event_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_cuda_stream_compared_with_stream_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_cuda_stream_context_manager1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_cuda_stream_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_disable_saved_tensors_hooks_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_disable_saved_tensors_hooks_prev_disabled_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_graph_break_inlining_autocast_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_graph_break_inlining_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_inactive_context_graph_break_local_nullctx_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_inactive_context_graph_break_stack2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_nested_generic_context_manager_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_nested_generic_context_manager_with_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_nested_grad_mode_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_no_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_return_context_manager_with_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_T_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_add_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_call_dict2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_call_dict3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_callable_class_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_callable_lambda_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_callable_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_class_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_cls_eq_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_cls_hasattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_cls_is_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_complex_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_constant1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_constant2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_constant3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_constant4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_context_wrapping_nested_functions_no_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_cublas_allow_tf32_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_custom_dict_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_default_dict_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_default_dict_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_default_dict_tuple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_del_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_device_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_fromkeys_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_mutable_map_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_ops_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_param_keys_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_values_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_distributed_is_available_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_distributed_is_initialized_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dtype_compare_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_fstrings1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_fstrings2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_get_autocast_gpu_dtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_get_default_dtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_get_device_properties_tensor_device_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_globalfn_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_import1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_indexed_range_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_indirect3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_inline_jit__unwrap_optional_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_inline_script_if_tracing_fn_with_default_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_inline_softmax_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_any_autocast_enabled_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_complex_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_contiguous_memory_format_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_floating_point_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_fx_tracing_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_islice_chain_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_itertools_chain_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_itertools_chain_from_iterable_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_itertools_product_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_len_constant_misc_iterables_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_clear_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_convert_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_expand_lhs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_reversed_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_listarg2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_listarg4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_load_global_bool_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_mT_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_math_radians_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_mean_sum_np_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_methodcall1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_methodcall3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_min_max_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_namedtuple_defaults_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_namedtuple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_namedtuple_hasattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_namedtuple_user_methods_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_ndarray_builtin_functions_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_ndarray_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_ndarray_transpose_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_ndim_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_no_recompile_inner_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_no_recompile_inner_lambda_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_not_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_np_constant_collections_as_input_int_or_float_float_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_np_constant_collections_guards_float_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_np_constant_collections_guards_int_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_np_iinfo_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_number_method_method_as_integer_ratio_num_type3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_number_method_method_conjugate_num_type2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_number_method_method_conjugate_num_type4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_number_method_method_is_integer_num_type6_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_numpy_attributes_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_numpy_dtype_argument_to_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_numpy_dtype_call_in_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_numpy_fft_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_numpy_linalg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_numpy_meshgrid_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_obj_eq_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partial_across_graph_break_uninvoked_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___annotations___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___class___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___code___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___dir___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___eq___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___ge___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___get___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___getattribute___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___globals___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___gt___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___init___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___init_subclass___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___kwdefaults___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___lt___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___module___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___qualname___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___reduce___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___reduce_ex___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___sizeof___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_lambda_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_recompilation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_torch_op_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_torch_op_kwarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_udf_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_udf_kwarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_udf_kwarg_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_pop_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_pos_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_pow_int_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_promote_types_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_rand_inlined_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_range1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_range_length_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_range_with_slice_index_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_reduce_with_initial_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_reduce_with_none_initial_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_reduce_with_single_with_initial_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_return_dict2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_return_multiple_numpy_ndarray_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_return_tuple2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_set_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_set_intersection_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_shape2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_slice3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_slice4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_slice5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_sum_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_sum_shortcut_with_start_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_sum_with_start_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_len_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_new_with_size_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_size_indexed_by_symint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_type4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_type5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_torch_from_numpy_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_truth_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tuple2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tuple_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tuple_sorted_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_unary_fold_op_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_unpack1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_unpack2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_unpack3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_unpack_ex1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_unpack_ex3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_viamethod_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_312_binary_slice_with_graph_break1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_T_tensor_attribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_add_to_set_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_anomaly_aot_autograd_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_aot_autograd_propagate_unbacked_symints_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_assert_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_assert_size_stride_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_auto_functionalize_can_with_default_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_auto_functionalize_can_with_none_return_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_auto_functionalize_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_auto_functionalize_on_view_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_auto_functionalize_optional_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_auto_functionalize_tensorlist_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_auto_functionalize_with_returns_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_backend_match_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_backend_match_guard_multi_threads_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_boolarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_builder_for_class_with_metaclass_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_builtin_abs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_builtin_subclasses_as_method_on_var_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_call_parent_non_class_methods_from_child_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_callpacked_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_can_auto_functionalize_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cat_unbacked_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_catch_watchings2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cell_output2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_class_has_instancecheck_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_clone_sparse_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_closure_out_of_scope_cell_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_closure_out_of_scope_cell_with_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_closure_with_mutation_and_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_compare_shapes_eq_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_compare_shapes_neq_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_compare_shapes_tuple_eq_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_compilation_metrics_size_limit_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_compile_profiler_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cond_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cond_export_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cond_nested_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_conditional_list_comp_in_context_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_config_getattr_default_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_contains_dunder_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cross_entropy_loss_fancy_ctor2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cross_entropy_loss_simple_ctor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cuda_set_device_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_default_dtype_change_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_deque_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_deterministic_algorithms_mutated_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dict_guard_on_keys_order_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dict_namedtuple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dict_order_keys_modules_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dict_order_keys_tensors_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dict_reconstruct_keeps_original_order_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dictcomp_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dunder_methods_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_duplicate_graph_break_log_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dynamic_one_hot_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dynamo_cache_invalidate_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dynamo_cache_move_to_front_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dynamo_compiling_fake_tensor_to_vararg_int_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_empty_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_enum_as_dict_key_with_overloaded_str_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_enum_guards_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_enum_no_graphbreaks_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_error_on_nested_fx_trace_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_error_on_recompile_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_flat_name_to_original_fqn_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_fn_hasattr__name__1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_fn_hasattr__name__2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_fn_hasattr__name__3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_funcname_cache_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_generate_trivial_abstract_impl_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_get_cache_entry_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_getset_descriptor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_grad_none_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_graph_break_compilation_metrics_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_graph_break_compilation_metrics_on_failure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_graph_break_correctly_when_passing_numpy_ndarray_to_torch_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guard_failure_fn2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guard_failure_fn_shape_control_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guard_size_oblivious_backed_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guard_size_oblivious_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guards_cse_pass_multiple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guards_strip_function_call_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_id_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_if_cond_nn_mod1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_if_cond_nn_mod3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inference_mode_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inline_closure_not_loaded_by_parent_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inline_dict_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inline_dict_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inline_func_jump_on_tensor_condition_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inline_list_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inline_local_dict_clear_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inplace_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inplace_view_on_graph_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inspect_signature_bind_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_int_shape_comparisons_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_int_shape_inplace_binops_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_intermediary_tensor_grad_access_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_interpolate_propagate_real_tensors_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_invalid_args_builtin_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_is_floating_point_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_is_tensor_like2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_is_tensor_like_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_item_changes_new_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_iter_set_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_iter_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_accumulate_symint_default_sum_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_accumulate_tensors_default_sum_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_accumulate_tensors_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_groupby_pure_python_default_identify_func_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_infinite_count_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_infinite_repeat_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_repeat_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_large_reduction_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_linear_module_free_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_list_append_return_none_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_list_iadd_side_effect_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_list_iadd_with_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_list_slice_mul_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_listcomp_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_load_fast_and_clear_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_map_with_quantization_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_mark_dynamic_with_ranges_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_module_complex_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_module_deepcopy_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_module_not_callable_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_namedtuple2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_closure_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_function_resuming_with_correct_globals_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_optimize_decorator_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_optimize_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_optimize_run_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_sequential_try_with_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_wraps_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nn_functional_reduction_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nn_module_getattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nn_sequential_invocation_reposition_indices_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_no_guard_for_unused_sym_node_fstring_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_no_raise_guard_partial_constraint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nonzero_static_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_as_global_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_fallback_on_eager_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_force_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_gt_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_ndarray_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_non_torch_dtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_random_config_to_numpy_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_readonly_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_recompilation_scalar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_take_along_axis_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_tolist_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_torch_operators_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_variable_isinstance_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_with_builtin_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_object_classmethod_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_object_staticmethod_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_onnx_shape_as_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_optimize_on_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_optree_graph_break_message_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_ordered_dict_alias_reconstruct_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_os_environ_get_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_os_environ_set_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_out_variants_with_resizing_on_graph_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_out_variants_with_resizing_on_graph_inputs_with_dynamic1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_out_variants_with_resizing_on_graph_inputs_with_dynamic_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_outside_linear_module_free_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_packaging_version_parse_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_pair_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_param_shape_binops_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_parameter_free_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_pt2_compliant_overload_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_pure_python_accumulate_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_py_guards_mark_dynamic_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_python_slice_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_raise_guard_full_constraint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_raise_guard_partial_constraint_across_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_raise_guard_partial_constraint_no_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_raise_on_backend_error_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_raises_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_range_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_range_with_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_real_imag_tensor_attribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_recompile_message_on_parameter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_recompile_on_global_state_change_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_recursive_inline_list_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_release_input_memory_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_repro_graph_breaks_in__get_item_by_idx_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_restore_graphstate_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_sample_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_scalar_tensor_is_equivalent_to_symint_argument_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_scalar_tensor_is_equivalent_to_symint_list_argument_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_set_aliasing_recompiles_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_setattr_mutation2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_setattr_mutation3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_and_tuple_equality_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_equal_empty_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_equal_evaluate_expr_divisible_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_equal_evaluate_expr_refinement_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_equal_evaluate_expr_replacement_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_equal_runtime_assert_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_no_recording_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_int_inplace_binops_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_side_effects_codegen_update_mutated_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_simple_set_usage_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_size_dim_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_storage_return_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_str_format_assert2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_str_format_return2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_super_calling_with_metaclass_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_symint_as_device_kwarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_symint_as_device_kwarg_non_strict_export_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_sys_modules_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tagging_tensors_mix_used_unused_structure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_data_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_dict1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_dict2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_dict3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_interacts_with_numpy_ndarray_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_is_contiguous_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tolist_kd_dynamic_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tolist_kd_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tolist_scalar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_compile_ctx_on_forward_and_training_step_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_cudnn_is_acceptable_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_device_python_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_guards_stack_frame_register_inlining_deep_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_guards_stack_frame_register_inlining_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_nn_parameter_isinstance_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_package_working_with_trace_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_seed_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_size_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_size_numel_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_variable_hasattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tracing_nested_py_tree_tuples_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tracing_py_tree_tensor_subclass_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tuple_from_tuple_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tuple_hasattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tuple_mul_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_type_copy_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_typing_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_typing_union_and_optional_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_typing_variable_isinstance_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_unbacked_symint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_unhandled_exception_in_dynamo_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_unpack5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_unpack_tensor_shape_mismatch_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_defined_binop_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_defined_class_python_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_defined_setattr1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_function_variable_supports_function_argument_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_function_variable_supports_type_abcmeta_argument_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_getattr1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_getattr2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_getattribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_usr_cls_classmethod_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_variable_tracker_recursively_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_version_ci_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_write_to_closures_in_inlining_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_yield_from_in_a_loop_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_yield_from_user_stop_iteration_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_yield_send_to_subgenerator_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_Size_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_add_sub_alpha_out_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_batch_encoding_clone_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_batchnorm_e2e_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_changing_stride_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_class_member_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_compile_complex_conj_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_const_dict_keyerror_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_data_attr_mutation_after_saved_for_bw_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dataclass_init_with_default_factory_with_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dedup_global_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dict_tag_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dynamic_shapes_float_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dynamic_shapes_right_side_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_ellipsis_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_embedding_backward_broadcasting_decomp_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_empty_list_contains_with_jump_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_ephemeral_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_error_return_without_exception_set_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_exec_wildcard_import_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_for_loop_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_get_parameter_dtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_grad_mode_carrying_correct_state_after_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_graph_break_on_jit_isinstance_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_graph_break_unsupported_fake_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_guard_fail_nested_tuple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_guard_fail_tensor_bool_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_guard_ordering_shape_fail_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_hf_bigbird_unsqueeze_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_hf_classinstantier_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_hf_gelu_inline_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_hf_model_output_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_hf_xsoftmax_inference_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_iadd_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_indexing_with_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_is_make_fx_tracing_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_is_symbolic_tracing_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_issue111522_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_issue111918_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_issue1466_size_aot_autograd_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_jit_trace_errors_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_kwargs_out_list_variable_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_list_aliasing_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_list_index_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_maml_item_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_many_views_with_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_map_with_multiple_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_megablocks_moe_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_merge_criteria_processor_list1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_multi_import_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_named_buffers_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_negative_floor_div_solve_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_nested_while_loop_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_nn_module_stack_bc_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_nonconst_issubclass_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_nullcontext1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_nullcontext2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_numpy_tobytes_no_error_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_omegaconf_listconfig_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_optim_state_references_cleared_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_optimized_deepcopy_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_optimized_module_training_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_overlapping_inputs_with_dynamic_shapes_error_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_partitioner_cse_respects_mutation_boundaries_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_primtorch_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_primtorch_no_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_recursive_map_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_reformer_eval_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_reformer_train_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_relative_import_no_modulename_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_requires_grad_guards_with_grad_mode1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_rewrite_assert_noop_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_rewrite_assert_with_msg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_rng_state_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_setattr_requires_grad_graph_breaks_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_setitem_boolean_mask_diff_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_setitem_tuple_boolean_mask_diff_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_sigmoid_out2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_size_typematch_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_slice_into_list_mutable_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_sort_out2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_sort_out_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_split_with_sizes_aot_autograd_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_stk_sdd_is_transposed_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_stop_iteration_reconstruct_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_sub_alpha_scalar_repro_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_super_in_staticmethod_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_swin_base_tensor_attr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_data_kwarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_item_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_set_data_backend_aot_eager_func_name_func1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_set_data_backend_aot_eager_func_name_func2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_set_data_backend_aot_eager_func_name_func3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_set_data_backend_eager_func_name_func1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_set_data_backend_eager_func_name_func2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_set_data_backend_inductor_func_name_func1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_set_data_backend_inductor_func_name_func2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_set_data_backend_inductor_func_name_func3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_torch_ops_aten_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_torch_tensor_ops_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_torch_variable_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_trace_functional_tensor_with_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tuple_enum_as_key_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_udf_classes_reconstruction_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_unpack_hooks_dont_run_during_tracing_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_user_ctor_ctx_manager_custom_init_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_user_ctor_ctx_manager_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_user_defined_object_callable_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_validate_model_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_weakref_del_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_weakref_reconstruct_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_while_loop_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_while_loop_graph_break_inside_call_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_access_by_keys_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_children_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_constloop_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_conv_call_super_forward_directly_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_conv_transpose_call_forward_directly_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_conv_transpose_call_super_forward_directly_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_densenet_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_enumvalues_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_fnmembercmp1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_fnmembercmp2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_generation_tag_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_iseval1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_istraining1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_lazy_module2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_lazy_module4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_lazy_module6_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_module_attribute_precedence_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_module_call_module_with_static_forward_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_module_name_string_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_module_static_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_moduledict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_modulelist_custom_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_modulelist_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_modulelist_nested_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_modulemethod1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_modulemethod2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_named_children_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_nn_moduledict_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_parameters1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_parameters2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_self_mutating1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_sequential_with_duplicated_module2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_simple_torch_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_submodules1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_super1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_torch_function_with_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_byte_tensor_does_not_crash_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_capture_symbolic_tracing_simple_within_fake_mode_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_capture_symbolic_tracing_within_fake_mode_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_raise_user_error_on_mismatch_return_length_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_raise_user_error_on_missing_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_raise_user_error_on_non_list_operands_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dataclass_input_output_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_2_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_and_bypass_reorder_with_non_tensor_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_and_bypass_with_non_tensor_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_and_bypass_with_non_tensor_arg_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_and_bypass_with_non_tensor_output_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_and_bypass_with_non_tensor_output_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dynamic_slicing_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dynamic_slicing_invalid_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_empty_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_compare_optimize_with_make_fx_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_control_flow_with_getattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_decomp_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_dynamic_dim_cleanup_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_dynamic_dim_raise_on_compound_range_constraint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_graph_bypass_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_graph_bypass_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_graph_with_complex_reorder_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_masking_with_no_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_mismatched_out_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_multi_dynamic_dim_constraint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_multi_dynamic_dim_unsafe_relationship_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_no_raise_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_pass_arg_by_name_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_pass_arg_by_name_star_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_persist_assert_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_preserve_constraints_as_metadata_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_preserves_nn_module_stack_for_get_attr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_raise_guard_full_constraint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_raise_on_relationship_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_shape_control_flow_1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_specialized_int_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_symbolic_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_args_with_default_float_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_args_with_default_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_cond_branches_calling_methods_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_cond_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_dict_values_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_free_function_and_class_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_method_on_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_method_on_module_invoke_twice_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_not_none_control_flow_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_not_none_control_flow_pos_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_functools_wrapped_fn_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_kwargs_with_default_None_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_kwargs_with_default_float_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_kwargs_with_default_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_kwargs_with_default_tuple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_map_cond_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_stack_trace_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_symbool_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_input_container_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_invalid_input_global_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_invalid_input_unused_nonlocal_ok_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_list_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_list_unpack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_multiple_outputs_op_with_evaluator_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_no_tensor_computation_2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_no_tensor_computation_2_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_predispatch_with_for_out_dtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_predispatch_with_for_out_dtype_nested_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_predispatch_with_higher_order_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_preserve_fx_node_metadata_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_preserve_fx_node_metadata_inline_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_preserve_fx_node_metadata_recompile_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_remove_redundant_dynamic_dim_in_error_message_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_retracibility_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_retracibility_nested_list_out_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_round_dynamic_shapes_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_sym_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_symbolic_tracing_within_fake_mode_with_constraints_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_symbolic_tracing_within_fake_mode_with_constraints_with_parameters_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_symbool_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_torch_inference_mode_ctx_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_zeroes_in_and_out_different_shape_on_test_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_zeroes_in_new_shape_scalar_out_permute_dupe_and_bypass_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_zeroes_in_new_shape_scalar_out_permute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_control_flow1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_dynamic_getitem_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_extended_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_graph_break_on_item_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_indirect_unsupported3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_multigraph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_restore_state_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume_paths_join_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume_with_no_grad1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_stack_state1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_start1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_start4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_tuple_iterator_mutate_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_access_module_attr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_constants_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_global_num_adds_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_input_num_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_tracked_nested_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_untracked_global_nested_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_branches_no_arguments_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_free_variable_in_both_branches_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_graph_break_in_one_branch_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_source_fn_stack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_subgraph_name_is_valid_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_fallback_on_graph_break_simple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_fallback_on_python_primitives_output_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_make_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_lowers_to_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_side_effect_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_source_fn_stack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_subgraph_name_is_valid_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_modules_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_nested_wrap_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_no_freevars_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_output_with_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_register_subclass_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_same_freevar_twice_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_del_existing_attr_global_obj_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_del_existing_attr_nonlocal_obj_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_in_body_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_local_list_append_no_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_mutate_global_num_builtin_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_mutate_global_num_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_mutate_nonlocal_num_builtin_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_mutate_nonlocal_num_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_mutate_nonlocal_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_nested_nonlocal_list_append_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_nonlocal_list_append_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_set_new_attr_global_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_set_new_attr_global_obj_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_vmap_multiply_scalar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_kwarg_default_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_kwarg_default_if_branch_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_kwarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_kwarg_only_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_kwarg_recompile_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_pytree_args_nested_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_pytree_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_source_fn_stack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_subgraph_name_is_valid_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_functional_call_disable_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_functional_call_disable_inline_nn_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_capture_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_freevar_python_scalar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_non_tensor_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_pytree_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_recompile_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_with_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_with_side_effect_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jacfwd_randomness_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jacrev_two_tensors_argnums_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jvp_disable_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jvp_has_aux_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jvp_jvp_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jvp_two_tensors_disable_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jvp_two_tensors_has_aux_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_linearize_jvp_fn_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vjp_disable_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vjp_has_aux_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vjp_multiple_outputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vjp_multiple_outputs_python_struct_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_disable_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_free_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_multiple_invocation_in_dims_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_multiple_outputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_new_tensor_unused_in_body_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_over_vmap_captured_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_previous_illegal_op_no_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_pytree_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_recompile_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_side_effects_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_two_inputs_tuple_in_dims_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_with_conditional_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_with_graph_break_2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_with_graph_break_lambda_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_alias_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_aot_autograd_expand_mutation_backwards_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_aot_autograd_expand_mutation_error_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_aot_autograd_raises_invalid_leaf_set_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_aot_sequence_nr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_arg_dupe_via_dynamo_recompiles_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_arg_dupe_via_dynamo_recompiles_many_with_global_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_data_ptr_access_copy_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_data_ptr_access_fails_in_backward_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer_with_retain_or_create_graph1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_double_backward_errors_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_eager_sequence_nr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_grad_inputs_alias_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_multiple_aot_autograd_calls_dupe_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_negative_testing_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_nn_parameter_construction_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_split_with_sizes_aot_autograd_cleans_up_traceback_meta_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesTestSDPA::test_graph_break_SDPAParams_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesTestSDPA::test_input_SDPAParams_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesTestSDPA::test_returns_SDPAParams_dynamic_shapes 2024-08-08T21:23:05.2188535Z 2024-08-08T21:23:09.0013535Z Running dynamo/test_dynamic_shapes 2/2 ... [2024-08-08 21:23:09.000676] 2024-08-08T21:23:09.0018291Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'dynamo/test_dynamic_shapes.py', '-m', 'not serial', '--shard-id=2', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:23:09.001193] 2024-08-08T21:29:45.5012189Z 2024-08-08T21:29:45.5014006Z dynamo/test_dynamic_shapes 2/2 was successful, full logs can be found in artifacts with path test/test-reports/dynamo.test_dynamic_shapes_2.2_c95727dd111c647a_.log 2024-08-08T21:29:45.5588860Z Running 815 items in this shard: test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_float64_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_graph_break_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_sdpa_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_context_wrapping_grad_mode_decorator_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_context_wrapping_set_grad_enabled_nested_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_cuda_event_method_create_stream_outside_of_compile_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_cuda_stream_across_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_cuda_stream_compared_with_constant_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_cuda_stream_context_manager2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_disable_saved_tensors_hooks_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_disable_saved_tensors_hooks_prev_disabled_nested_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_generic_context_manager_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_generic_context_manager_with_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_generic_ctx_manager_with_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_grad_mode_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_inactive_context_graph_break_local_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_inactive_context_graph_break_local_nullctx2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_inactive_context_graph_break_stack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_is_autocast_cpu_enabled_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_return_context_manager_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_torch_profiler_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_add__dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_addcdiv__dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_addcdiv_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_build_list_unpack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_call_dict1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_call_dict4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_call_dict5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_callable_builtin_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_callable_torch_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_chunks1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_compare_constant_and_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_const_tuple_add1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_const_tuple_add2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_default_dict_constr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_default_dict_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_default_dict_lambda_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_default_dict_set_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_deque_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_device_constant_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_copy_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_id_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_keys_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_sorted_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_tuple_lazy_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dict_update_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_dtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_elipsis_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_finfo_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_flat_param_same_storage_size_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_float_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_fn_with_self_set_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_fstrings3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_fstrings4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_fstrings5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_fstrings6_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_funcdef_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_functools_partial_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_get_calculate_correct_fan_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_get_privateuse1_name_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_globalmodule_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_globalvar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_in_not_in_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_index_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_indirect1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_indirect2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_inline_jit_annotations_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_inline_lru_cache_fn_with_default_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_inline_with_default_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_inner_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_contiguous_frame_counts_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_in_onnx_export_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_integer_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_not_null_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_quantized_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_is_sparse_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_itertools_combinations_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_jit_annotate_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_len_constant_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_len_constant_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_len_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_add_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_add_then_mutate_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_compare_polyfill_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_index_with_constant_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_slice_assignment_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_sorted1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_sorted2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_list_truth_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_listarg1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_listarg3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_listarg5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_manual_seed_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_map_sum_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_methodcall2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_module_constant_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_ndarray_methods_returning_scalar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_ndarray_reshape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_non_inlined_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_np_constant_collections_as_input_int_or_float_int_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_np_finfo_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_number_method_method_as_integer_ratio_num_type0_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_number_method_method_bit_length_num_type1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_number_method_method_hex_num_type5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_numpy_random_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_numpy_size_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_obj_is_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_ordered_dict_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_as_input_UDF_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_as_input_partials_lambda_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_as_input_partials_mod_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_graph_break_reconstruct_args_and_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_graph_break_reconstruct_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_graph_break_reconstruct_mix_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_graph_break_reconstruct_mix_no_source_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___builtins___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___call___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___closure___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___defaults___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___delattr___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___dict___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___doc___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___format___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___hash___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___le___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___name___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___ne___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___new___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___repr___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___setattr___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___str___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr___subclasshook___dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr_func_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_attr_keywords_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_hasattr_set_attr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_partials_udf_kwarg_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_rand_tensor_partial_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_range2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_range_with_index_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_reduce_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_reduce_with_single_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_return_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_return_numpy_ndarray_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_return_tuple1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_set_difference_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_set_isdisjoint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_set_keys_view_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_set_union_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_set_update_bytecode_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_set_update_list_with_duplicated_items_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_shape1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_slice1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_slice2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_slice6_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_sliced_range_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_startswith_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_sum_shortcut_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_sum_shortcut_with_start_kwarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_sum_with_start_kwarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_symbool_to_int_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_element_size_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_is_complex_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_new_with_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_type2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_type3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tensor_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_to_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_torch_distributions_functions_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_torch_size_hasattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_transpose_for_scores_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tuple1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_tuple_iadd_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_unary_fold_op_seq_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_unpack_ex2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_unpack_mutable_map_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_viatorch_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFunctionTests::test_zip_longest_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_312_binary_slice_with_graph_break2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_add_sizes_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_any_all_symnode_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_assigning_function_to_class_attribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_assigning_function_to_object_attribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_auto_functionalize_self_as_mutate_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_backward_deterministic_mode_mismatch_warning_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_build_tuple_unpack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_builtin_isinstance_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_builtin_str_on_user_defined_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_builtin_subclasses_as_method_on_class_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cannot_trace_mark_dynamic_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cannot_trace_mark_dynamic_safe_unreached_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cast_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_catch_watchings1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cell_output1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_class_duner_mro_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_closure_out_of_scope_cell_with_cond_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_closure_recompiles_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_compare_shapes_tuple_neq_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_compare_shapes_with_constant_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cond_export_single_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cond_side_effects_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cond_with_quantization_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_config_obj_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_const_dict_variable_python_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_constant_getattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cpp_extension_recommends_custom_ops_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cross_entropy_loss_fancy_ctor1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_cse_dict_guards_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_custom_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_custom_iter_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_custom_keys_iter_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_custom_module_free_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dataclass_fields_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dataclass_local_hasattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_default_args_device_dtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_defaultdict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_deque_append_left_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_derpy_nn_module_usage_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_descriptor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dict_guard_on_keys_order2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dict_mutation_side_effect_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dict_order_keys_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dict_subclass_cannot_be_initialized_in_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_disable_flag_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dtypes_no_graphbreaks_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dunder_new_function_inlining_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dynamo_min_operator_with_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_dynamo_reset_clears_cache_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_enum_as_dict_key_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_fold_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_frozen_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_frozenset_torch_func_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_function_annotation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_generate_tensor_from_list_of_numpy_primitive_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_get_attr_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_get_custom_tensor_attribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_get_device_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_get_instruction_source_311_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_getattr_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_grad_non_none_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_grad_state_mutated_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guard_failure_fn_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guard_failure_fn_tensor_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guard_function_builder_with_cse_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guard_sym_node_fstring_when_used_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_guards_cse_pass_single_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_hasattr_nn_module_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_hash_getitem_slice_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_id_guarded_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_id_guarded_object_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_id_of_nn_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_if_cond_nn_mod2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_if_cond_user_defined_object2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_if_cond_user_defined_object3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_if_cond_user_defined_object_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inline_dict_function_passed_as_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inline_module_attr_dict_clear_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inline_user_defined_dict_attr_clear_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inplace_desugaring_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inplace_param_update_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_input_set_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_inspect_signature_bind_non_user_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_int_int_comparisons_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_int_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_int_neg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_int_shape_binops_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_is_compiling_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_is_floating_point2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_is_tensor2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_is_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_item_changes_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_item_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_accumulate_tensors_builtins_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_accumulate_tensors_user_defined_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_groupby_pure_python_key_func_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_infinite_cycle_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_itertools_infinite_repeat_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_list_hasattr1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_list_hasattr2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_list_iterator_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_list_mul_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_mandelbrot_numpy_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_map_side_effects_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_mark_static_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_matmul1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_min_max_over_iterable_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_module_dunder_dict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_named_parameters_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_namedtuple1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_namedtuple3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nan_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_sequential_try_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_sequential_try_with_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nested_sequential_with_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_new_with_int_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nn_module_getattribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_nn_sequential_invocation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_no_error_on_nested_fx_trace_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_no_raise_guard_partial_constraint_across_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_non_pt2_compliant_ops_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_not_dynamic_scope_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numel_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_array_of_arrays_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_int_constant_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_min_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_ndarray_graph_break_with_multiple_outputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_ndarray_works_with_builtin_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_no_raise_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_size_attr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_subdtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_ufunc_out_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_ufunc_out_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_numpy_unique_f16_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_object_setattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_ordered_dict_move_to_end_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_out_variant_custom_op_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_parsing_sdpa_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_patched_builtin_functions_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_pt2_compliant_ops_are_allowed_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_raise_guard_indirect_full_constraint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_raises_importerror1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_raises_importerror2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_rand_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_reconstruct_set_across_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_recursive_tensor_attribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_release_module_memory_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_release_scope_memory_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_repeat_interleave_graphbreaks_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_return_dict_with_graph_break_and_update_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_return_nested_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_runtime_assert_replacement_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_scalar_tensor_is_equivalent_to_int_list_argument_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_sequential_module_free_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_set_custom_tensor_attribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_setattr_mutation1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_equal_constructor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_equal_create_symbolic_sizes_strides_storage_offset_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_equal_unbacked_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_env_recorded_function_fallback_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_int_comparisons_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_shape_unpack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_size_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_slice_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_source_non_input_grad_access_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_str_format_assert1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_str_format_return1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_stride_dim_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_super_after_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_sym_constrain_range_on_replaced_unbacked_symbol_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_symint_as_device_kwarg_multi_gpu_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tagging_tensors_simple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_build_list_unpack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_ctor_list_of_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_dot_grad_no_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_hasattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_item_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_item_no_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_layout_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tensor_types_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tolist_0d_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tolist_1d_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tolist_float_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_top_package_import_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_check_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_check_is_size_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_check_symbolic_shape_rel_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_cuda_is_available_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_cudnn_is_acceptable_bad_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_distributions_lazy_property_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_dtype_python_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_dynamo_codegen_pow_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_generator_set_state_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_objects_as_keys_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_torch_size_numel_dynamic_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_trace_ndarray_frame_2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_trace_ndarray_frame_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tracing_nested_py_tree_dicts_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tracing_nested_py_tree_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tracing_nested_py_tree_mixed_all_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tracing_py_tree_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tracing_tree_map_only_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tuple_iadd_with_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_tuple_mul_with_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_typing_typevar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_unbacked_auto_functionalize_op_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_unhandled_exception_in_dynamo2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_unpack4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_update_locals_and_stack_uses_shared_cache_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_defined_class_name_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_defined_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_defined_setattr2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_function_variable_supports_enum_argument_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_user_property_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_usr_cls_staticmethod_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_validate_outputs_unbacked_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_variable_access_in_exception_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_with_builtin_type_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_yield_from_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_yield_gen_and_from_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_abc_setattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_addr_alpha_beta_out_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_as_strided_on_base_with_mutation_works_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_as_strided_on_existing_view_banned_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_attached_attribute_in_dir_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_autograd_function_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_avoid_dupe_specialization_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_batch_norm_act_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_bigbird_unsqueeze_inplace_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_boxes_len_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_build_map_unpack_with_call_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_call_finally_opcode_python_3_8_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_call_finally_python_3_8_2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_call_finally_python_3_8_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_chunk_reformer_ff_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_contains_range_constprop_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_convert_boxes_to_pooler_format_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_copy_weird_strides_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_create_rand_mask_from_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dalle2_maybe_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_deferred_runtime_asserts_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_delattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_delattr_raises_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_delsubscr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_delsubscr_raises_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_detectron2_instances_cat_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dict_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dict_list_values_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dict_subclass_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_disabling_unpack_hooks_within_compiled_region_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_do_paste_mask_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dont_aggressively_write_assert_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dropout_inline_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dynamic_shape_disable_duck_size_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dynamic_shapes_double_not_equal_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dynamic_shapes_implicit_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_enum_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_exception_in_dynamo_handling_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_exec_import_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_flash_attn_backward_mixed_strides_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_for_loop_graph_break_before_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_fsdp_set_input_mutation_applied_when_input_gets_no_gradients_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_function_in_skipfiles_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_functools_wraps_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_gan_repro_trying_to_backward_through_the_graph_a_second_time_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_generator_dealloc_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_global_fn_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_grad_references_cleared_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_guard_default_device_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_guard_with_tuple_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_hasattr_builtin_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_hf_t5_forward_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_hf_xsoftmax_training_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_inductor_no_recursionerror_on_for_loops_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_inference_mode_dynamic_shapes_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_inplace_unsqueeze_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_intermediate_leaf_requires_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_invalid_seq_unpack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_isinstance_dtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_isinstance_storage_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_issue114171_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_issue126128_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_issue175_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_jit_script_defaults_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_list_index_not_found_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_list_index_tensor_unsupported_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_list_reverse_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_list_self_reference_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_longformer_chunk_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_longtensor_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_maml_no_item_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_many_overlapping_inputs_does_not_explode_guards_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_maybe_multiply_symint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_merge_criteria_processor_list2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_module_in_skipfiles_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_modules_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_multi_dot_import_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_negative_shape_guard_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_nn_module_callable_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_nn_module_property_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_nn_parameter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_nn_parametrize_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_no_grad_inline_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_norm_dtype_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_not_rewrite_assert_for_other_errors_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_numpy_not_ndarray_recompiles_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_odict_get_item_index_name_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_out_none_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_out_overload_non_contiguous_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_output_aliases_intermediate_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_partially_initialized_module_property_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_pointless_graph_removal_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_randint_out_dynamic_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_reformer_min_chunk_len_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_reformer_sorting_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_reinplacing_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_relative_import_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_requires_grad_guards_with_grad_mode2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_restricted_list_subclass1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_restricted_list_subclass2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_restricted_list_subclass3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_rewrite_assert_dont_change_bytecode_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_rewrite_assert_with_non_string_msg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_rewrite_assert_without_msg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_seq_append_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_sigmoid_out_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_slicing_dynamic_shape_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_slicing_dynamic_shape_setitem_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_specialized_stride_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_staticmethod_allow_in_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_string_format_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_subclass_graph_output_repro_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_isinstance_tuple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_random_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_set_data_backend_eager_func_name_func3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_split_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tensor_uniform_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_threading_local_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_tokenization_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_torch_tensor_ops_no_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_unbacked_arange_in_bounds_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_unpack_hooks_can_be_disabled_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_unspecialized_nn_module_with_torch_variable_attribute_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_user_ctor_ctx_manager_custom_init_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_user_defined_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_vc_bumped_in_inference_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_vdd_duplicate_error_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_view_dtype_overload_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_weakref_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_with_on_graph_break_inst_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_with_on_graph_break_nested_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_basicmodule1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_basicmodule2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_call_fn_with_non_const_inputs_safe_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_cfgmod_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_conv_call_forward_directly_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_fnmember_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_forward_directly_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_hasattr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_intarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_iseval2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_isnonelayer_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_istraining2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_layerlist_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_lazy_module1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_lazy_module3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_lazy_module5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_lazy_module7_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_lazy_module_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_lazy_module_no_cls_to_become_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_module_class_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_module_comparison_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_module_forward_has_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_module_guard_name_is_valid_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_module_property_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_moduledict_custom_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_parameterdict_custom_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_parameterdict_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_parameters3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_parameters4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_parameters5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_seq_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_sequential_with_duplicated_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_stringmember_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_submodules2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_super2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_super_class_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_tensorlist_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_unsupportedmethod_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_unsupportedmodule_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesNNModuleTests::test_viamodulecall_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_access_class_method_from_user_class_attr_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_access_class_method_from_user_class_builtin_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_free_variables_overlapping_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_op_param_buffer_lifted_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_raise_user_error_on_branch_args_mismatch_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_raise_user_error_on_branch_return_multiple_tensors_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_raise_user_error_on_branch_return_non_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_raise_user_error_on_mismatch_return_tensor_meta_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_raise_user_error_on_non_tensor_operands_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_raise_user_error_on_unsupported_pred_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_cond_supported_pred_types_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_constraint_violation_error_messages_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dict_return_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dict_return_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_and_bypass_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_and_bypass_reorder_with_non_tensor_arg_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_and_bypass_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dupes_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dynamic_slicing_simple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_dynamo_list_index_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_enforce_equalities_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_cond_in_aten_symbolic_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_decomp_asserts_bad_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_defaults_ok_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_dynamic_control_flow_error_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_dynamic_dim_not_1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_dynamic_dim_range_constraint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_fast_binary_broadcast_check_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_graph_with_complex_reorder_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_graph_with_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_graph_with_list_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_identity_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_mark_dynamic_conflict_dynamic_dim_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_meta_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_meta_val_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_mismatched_out_2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_mismatched_out_2_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_mismatched_out_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_module_specify_constraints_signature_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_nn_module_stack_patched_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_no_tensor_computation_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_preserve_constraints_as_metadata_scalar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_raise_guard_partial_constraint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_args_and_empty_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_args_with_default_None_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_args_with_default_tuple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_builtin_op_on_assume_constant_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_cond_dynamic_shape_pred_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_cond_with_closed_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_free_function_and_class_method_multiarg_diff_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_free_function_and_class_method_multiarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_free_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_in_unspecialized_nn_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_list_nonzero_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_list_nonzero_free_function_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_none_control_flow_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_none_control_flow_free_func_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_not_none_control_flow_free_func_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_not_return_const_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_constant_tuple_nonzero_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_functools_wrapped_method_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_kwargs_and_empty_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_map_zero_sized_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_map_zero_sized_tensor_suppress_errors_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_module_layer_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_nonzero_static_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_parameters_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_shallow_list_copy_with_side_effects_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_shallow_list_copy_wo_side_effects_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_export_with_wrapped_fn_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_exported_graph_serialization_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_func_return_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_func_return_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_fx_pytree_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_invalid_input_global_multiple_access_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_invalid_input_nonlocal_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_list_not_contains_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_list_unpack_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_map_cond_param_buffer_lifted_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_mixed_real_and_fake_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_nested_cond_op_param_buffer_lifted_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_no_tensor_computation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_no_tensor_computation_fail_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_not_functionalize_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_param_buffer_safe_from_mutation_recurse_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_param_buffer_safe_from_mutation_simple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_pre_dispatch_simple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_predispatch_with_higher_order_nested_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_preserve_fx_node_metadata_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_retracibility_dict_container_inp_out_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_sum_param_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_trivial_constraint_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_uncaptured_higher_order_op_error_not_suppresed_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_untracked_inputs_in_constraints_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_zeroes_in_and_out_different_shape_on_test_with_aten_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesExportTests::test_zeroes_in_new_shape_scalar_out_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_capi_call1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_capi_call2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_capi_call3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_control_flow2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_control_flow3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_control_flow4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_control_flow5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_dynamic_duck_size_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_dynamic_kwarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_dynamic_order_dependence_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_dynamic_zero_inference_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_enumerate_not_break_graph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_indirect_unsupported1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_indirect_unsupported2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_no_graph_break_on_item_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_pop_after_resume_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_restore_range_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_restore_range_iter_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume_freevars_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume_tuple_iterator_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume_with_no_grad2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_resume_with_no_grad3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_stack_state2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_start2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_start3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesSubGraphTests::test_tuple_iterator_return_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_global_num_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_numpy_number_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_tracked_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_untracked_global_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_untracked_nonlocal_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_capture_value_created_in_subgraph_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_branches_no_arguments_no_closure_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_pytree_operands_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_pytree_operands_with_non_tensor_leaves_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_side_effect_in_one_branches_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_cond_with_constant_pred_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_enum_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_error_message_sane_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_fallback_on_graph_break_complicated_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_flat_list_output_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_fn_with_kwargs_in_torch_ops_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_freevars_as_inputs_to_wrap_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_grad_source_fn_stack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_hooks_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_inlined_functions_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_internal_nonlocal_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_lift_tensor_constant_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_example_value_metadata_consistent_with_eager_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_multi_return_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_pytree_return_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_map_symint_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_nested_tuple_output_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_register_mode_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_return_captured_var_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_return_captured_var_used_multiple_times_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_return_captured_vars_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_del_existing_attr_global_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_del_existing_attr_nonlocal_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_mutate_global_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_mutate_global_tensor_builtin_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_mutate_global_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_mutate_nonlocal_tensor_builtin_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_set_existing_attr_global_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_set_existing_attr_global_obj_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_set_existing_attr_nonlocal_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_set_existing_attr_nonlocal_obj_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_set_new_attr_nonlocal_module_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_side_effect_set_new_attr_nonlocal_obj_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_symint_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_vmap_source_fn_stack_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_all_kwarg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_allow_local_assign_in_body_fn_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_kwarg_default_else_branch_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_kwarg_int_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_pytree_args_not_const_symint_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesHigherOrderOpTests::test_wrap_pytree_args_with_symint_constant_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_functional_call_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_functional_call_sequential_params_and_buffers_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_closure_scalar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_disable_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_fn_with_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_freevar_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_has_aux_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_over_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_two_tensor_all_grad_has_aux_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_grad_two_tensor_has_aux_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_hessian_argnums_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_hessian_disable_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_hessian_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jacfwd_disable_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jacfwd_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jacfwd_has_aux_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jacfwd_two_tensors_argnums_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jacrev_disable_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jacrev_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jacrev_has_aux_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jvp_freevar_python_scalar_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jvp_freevar_tensor_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jvp_simple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_jvp_two_tensors_disable_enable_disable_grad_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_linearize_disable_capture_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vjp_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_free_const_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_get_wrapped_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_kwargs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_multiple_invocation_out_dims_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_multiple_outputs_diff_dims_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_multiple_outputs_out_dims_tuple_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_new_tensor_implicit_via_op_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_new_tensor_in_body_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_over_vmap_two_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_recompile_different_config_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_recompile_same_config_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_recompile_with_randomness_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_side_effects_append_input_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_two_inputs_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesFuncTorchHigherOrderOpTests::test_vmap_with_graph_break_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_LSTM_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_aot_autograd_expand_mutation_functionalizes_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_aot_export_joint_simple_repro_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_aot_grad_mode_mutation_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_arg_dupe_via_dynamo_recompiles_many_args_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_arg_dupe_via_dynamo_recompiles_many_args_param_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_arg_dupe_via_dynamo_recompiles_many_args_param_non_tensor_arg_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_arg_dupe_via_dynamo_recompiles_many_args_param_non_tensor_arg_list_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_call_fn_with_non_const_inputs_aot_safe_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_call_fn_with_non_const_inputs_aot_unsafe_control_flow_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_call_fn_with_non_const_inputs_aot_unsafe_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_data_ptr_access_fails_in_forward_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer5_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer_with_retain_or_create_graph2_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer_with_retain_or_create_graph3_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer_with_retain_or_create_graph4_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_mutation1_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_negative_testing_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_requires_grad_fake_via_dynamo_recompiles_dynamic_shapes, test/dynamo/test_dynamic_shapes.py::DynamicShapesTestSDPA::test_intermediate_attr_access_SDPAParams_dynamic_shapes 2024-08-08T21:29:45.6135951Z 2024-08-08T21:29:49.4059674Z Running test_ops 4/4 ... [2024-08-08 21:29:49.405293] 2024-08-08T21:29:49.4064024Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_ops.py', '-m', 'not serial', '--shard-id=4', '--num-shards=4', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:29:49.405877] 2024-08-08T21:30:11.8007962Z 2024-08-08T21:30:11.8009913Z PRINTING LOG FILE of test_decomp 1/7 (test/test-reports/test_decomp_1.7_e987e43d390a110b_.log) 2024-08-08T21:30:11.8012812Z Test results will be stored in test-reports/python-pytest/test_decomp/test_decomp-86ac3fdb3ab1fe1f.xml 2024-08-08T21:30:11.8015067Z ============================= test session starts ============================== 2024-08-08T21:30:11.8017399Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-08T21:30:11.8019150Z cachedir: .pytest_cache 2024-08-08T21:30:11.8021360Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-08T21:30:11.8023652Z rootdir: /var/lib/jenkins/workspace 2024-08-08T21:30:11.8024665Z configfile: pytest.ini 2024-08-08T21:30:11.8026585Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-08T21:30:11.8028758Z collecting ... collected 8751 items 2024-08-08T21:30:11.8030066Z stepcurrent: Cannot find last run test, not skipping 2024-08-08T21:30:11.8707640Z Running 1228 items in this shard: test/test_decomp.py::TestDecompCUDA::test_cat_single_input_cuda, test/test_decomp.py::TestDecompCUDA::test_comprehensive_H_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_H_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_T_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive___getitem___cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive___getitem___cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive___getitem___cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive___radd___cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive___radd___cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive___radd___cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive___radd___cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rand___cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rmod___cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rmod___cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rmul___cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rmul___cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rmul___cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rmul___cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive___ror___cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive___ror___cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rpow___cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rpow___cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive___rsub___cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive__batch_norm_with_update_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive__native_batch_norm_legit_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive__segment_reduce_lengths_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive__segment_reduce_lengths_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive__softmax_backward_data_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive__softmax_backward_data_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive__unsafe_masked_index_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive__unsafe_masked_index_put_accumulate_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_abs_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_add_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addbmm_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addcdiv_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addcdiv_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addcmul_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addcmul_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addcmul_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addmm_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addmm_decomposed_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addmm_decomposed_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addmv_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addmv_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addr_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_addr_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_alias_copy_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_all_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_all_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_amax_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_amin_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_amin_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_angle_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_angle_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_any_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_any_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_any_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_arange_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_arange_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_argmax_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_argmin_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_argsort_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_argsort_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_argwhere_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_argwhere_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_copy_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_copy_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_scatter_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_scatter_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_scatter_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_asin_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_asin_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_asin_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atan2_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atan2_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atan_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atan_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atan_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atanh_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atanh_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atanh_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atleast_2d_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_atleast_3d_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_baddbmm_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_baddbmm_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bernoulli_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bfloat16_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bfloat16_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bincount_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_and_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_left_shift_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_left_shift_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_not_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_right_shift_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_xor_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_block_diag_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_block_diag_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bool_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_bool_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_broadcast_shapes_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_broadcast_tensors_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_broadcast_to_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_byte_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_byte_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cdouble_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ceil_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cfloat_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cfloat_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cfloat_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_chalf_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_chalf_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_char_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_char_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_char_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_char_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cholesky_inverse_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cholesky_solve_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cholesky_solve_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_chunk_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_clamp_max_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_clamp_min_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_clamp_min_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_clone_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_clone_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_clone_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_column_stack_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_column_stack_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_combinations_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_combinations_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_conj_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_conj_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_conj_physical_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_corrcoef_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_corrcoef_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_corrcoef_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cos_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cos_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cosh_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cosh_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_count_nonzero_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_count_nonzero_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cov_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cov_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cov_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cov_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cross_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cummax_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cummax_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cummin_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cummin_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cumprod_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cumprod_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cumprod_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cumsum_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_cumulative_trapezoid_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_diag_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_diagflat_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_diagflat_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_diagonal_copy_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_diagonal_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_diff_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_diff_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_diff_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_digamma_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_div_floor_rounding_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_div_floor_rounding_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_div_no_rounding_mode_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_div_no_rounding_mode_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_div_trunc_rounding_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_dot_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_dstack_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_dstack_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_dstack_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_empty_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_empty_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_empty_like_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_empty_like_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_empty_permuted_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_empty_permuted_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_empty_permuted_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_empty_permuted_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_empty_strided_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_eq_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_equal_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_equal_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_erfinv_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_exp_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_exp_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_expand_as_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_expand_copy_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_expand_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_expand_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_expand_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_expm1_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_eye_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_fft2_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_fft_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_fft_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_hfft2_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_hfft2_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_hfft_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifft2_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifft_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftn_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftn_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftn_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftshift_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftshift_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftshift_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftshift_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ihfft2_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ihfft_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ihfft_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ihfftn_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft2_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft2_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfftn_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fft_rfftn_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_flatten_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_flatten_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_flatten_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_flip_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_flip_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_flip_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fliplr_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fliplr_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fliplr_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_flipud_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_float_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_float_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_float_power_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_float_power_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_floor_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_floor_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_floor_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_floor_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_floor_divide_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_fmod_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_frexp_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_full_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_full_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_full_like_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_full_like_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_gather_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_gather_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_gcd_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_gt_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_half_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_half_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_heaviside_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_hsplit_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_hsplit_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_hsplit_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_hstack_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_hstack_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_hstack_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_hstack_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_i0_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_igamma_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_igammac_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_add_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_copy_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_copy_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_copy_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_copy_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_fill_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_fill_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_put_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_amax_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_amin_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_amin_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_mean_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_mean_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_prod_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_prod_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_index_select_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_inner_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_inner_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_inner_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_inner_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_int_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_int_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isclose_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isclose_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isfinite_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isin_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isin_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isinf_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isinf_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isinf_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isnan_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isnan_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isneginf_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isposinf_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isposinf_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isreal_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_isreal_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_item_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_2inputs_2outputs_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_4inputs_with_extra_args_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_binary_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_binary_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_binary_return_by_ref_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_unary_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_unary_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_kron_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_kthvalue_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_kthvalue_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_lcm_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ldexp_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_le_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_le_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_le_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_le_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_lgamma_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_cholesky_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_cond_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_cross_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_det_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_diagonal_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_diagonal_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_eig_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_eigvals_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_householder_product_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_ldl_factor_ex_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_lu_factor_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_lu_factor_ex_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_matrix_power_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_matrix_rank_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_matrix_rank_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_matrix_rank_hermitian_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_norm_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_norm_subgradients_at_zero_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_pinv_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_pinv_singular_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_qr_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_svdvals_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_tensorsolve_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_vander_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_vector_norm_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linspace_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linspace_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linspace_tensor_overload_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_linspace_tensor_overload_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log10_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log1p_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log1p_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log2_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log2_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log_normal_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log_softmax_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log_softmax_with_dtype_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_log_softmax_with_dtype_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logdet_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logical_not_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logical_or_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logical_or_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logical_xor_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logical_xor_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logit_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_tensor_overload_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_tensor_overload_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_tensor_overload_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_tensor_overload_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_long_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_long_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_long_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_lu_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mH_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mH_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mH_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mT_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mT_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mT_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_amax_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_amax_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_amin_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_amin_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_argmax_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_argmax_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_argmin_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_cumprod_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_cumsum_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_cumsum_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_fill_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_fill_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_log_softmax_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_mean_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_mean_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_mean_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_mean_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_normalize_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_normalize_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_prod_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_prod_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_prod_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_scatter_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_scatter_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_select_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_select_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_select_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_select_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_softmax_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_softmax_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_sum_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_var_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_masked_var_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_matmul_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_max_binary_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_max_binary_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_max_reduction_no_dim_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_max_reduction_no_dim_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_max_reduction_with_dim_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_maximum_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_maximum_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_maximum_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mean_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_median_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_meshgrid_list_of_tensors_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_meshgrid_list_of_tensors_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_meshgrid_list_of_tensors_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_meshgrid_variadic_tensors_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_min_binary_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_min_reduction_no_dim_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_min_reduction_with_dim_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_minimum_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_minimum_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_minimum_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mm_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mode_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_movedim_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_movedim_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_msort_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mul_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mv_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mvlgamma_mvlgamma_p_1_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mvlgamma_mvlgamma_p_1_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_mvlgamma_mvlgamma_p_5_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nanmean_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nanmean_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nansum_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nansum_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_narrow_copy_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_narrow_copy_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_narrow_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_narrow_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ne_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ne_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_empty_strided_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_empty_strided_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_empty_strided_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_full_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_full_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_full_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_ones_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_ones_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_ones_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_new_zeros_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_adaptive_avg_pool1d_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_adaptive_avg_pool2d_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_alpha_dropout_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_avg_pool1d_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_batch_norm_without_cudnn_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_binary_cross_entropy_with_logits_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_channel_shuffle_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_channel_shuffle_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_channel_shuffle_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_conv1d_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_conv2d_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_conv3d_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_conv_transpose2d_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_cosine_embedding_loss_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_cosine_similarity_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_cross_entropy_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_dropout2d_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_dropout_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_embedding_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_fractional_max_pool2d_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_fractional_max_pool2d_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_gelu_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_glu_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_grid_sample_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_hardshrink_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_hardswish_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_hardtanh_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_huber_loss_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_instance_norm_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_area_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_bilinear_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_linear_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_linear_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_nearest-exact_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_trilinear_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_kl_div_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_l1_loss_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_layer_norm_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_linear_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_local_response_norm_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_local_response_norm_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_margin_ranking_loss_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_pool1d_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_pool3d_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_unpool1d_grad_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_unpool2d_grad_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_unpool3d_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_mse_loss_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_mse_loss_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_multilabel_margin_loss_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_nll_loss_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_circular_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_constant_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_constant_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_constant_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_reflect_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_replicate_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_replicate_negative_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pairwise_distance_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pixel_shuffle_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pixel_shuffle_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pixel_unshuffle_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pixel_unshuffle_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_poisson_nll_loss_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_poisson_nll_loss_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_prelu_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_relu6_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_relu6_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_scaled_dot_product_attention_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_scaled_dot_product_attention_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softmin_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softmin_with_dtype_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softplus_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softsign_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softsign_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softsign_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softsign_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_tanhshrink_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_threshold_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_threshold_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_triplet_margin_with_distance_loss_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nonzero_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nonzero_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_nonzero_static_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_norm_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_norm_fro_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_normal_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_normal_in_place_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ones_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ones_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ones_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ones_like_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_outer_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_pca_lowrank_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_permute_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_pinverse_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_0_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_0_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_0_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_1_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_2_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_2_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_3_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_3_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_3_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_4_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_positive_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_positive_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_pow_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_pow_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_prod_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_prod_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_put_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_rad2deg_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_randint_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_randint_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_randint_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_randint_like_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_randint_like_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_randint_like_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_randn_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_randn_like_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ravel_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ravel_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_ravel_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_real_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_reciprocal_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_reciprocal_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_remainder_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_remainder_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_remainder_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_renorm_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_repeat_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_repeat_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_repeat_interleave_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_repeat_interleave_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_as_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_as_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_as_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_resize__cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_resize_as__cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_resize_as__cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_resolve_conj_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_resolve_neg_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_resolve_neg_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_roll_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_roll_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_rot90_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_round_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_round_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_round_decimals_0_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_round_decimals_3_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_round_decimals_neg_3_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_rsqrt_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_rsub_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scalar_tensor_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_add_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_add_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amax_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amax_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amin_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amin_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amin_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_mean_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_sum_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_sum_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_searchsorted_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_searchsorted_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_searchsorted_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_select_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_select_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_select_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_select_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_select_scatter_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sgn_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sigmoid_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_signal_windows_bartlett_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_signal_windows_general_hamming_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_signal_windows_hann_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_signal_windows_kaiser_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_signbit_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_signbit_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_signbit_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sinc_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sinc_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sinh_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sinh_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sinh_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_slice_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_slice_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_slice_scatter_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_slice_scatter_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_softmax_with_dtype_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sort_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sparse_mm_reduce_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_airy_ai_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_airy_ai_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_j0_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_j0_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_j0_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_j1_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_y0_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_chebyshev_polynomial_t_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_chebyshev_polynomial_t_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_chebyshev_polynomial_v_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_chebyshev_polynomial_v_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_entr_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_entr_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_erfcx_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_hermite_polynomial_h_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_hermite_polynomial_he_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_i0e_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_i0e_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_i1_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_i1_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_i1_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_log_ndtr_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_modified_bessel_k0_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_ndtr_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_ndtri_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_scaled_modified_bessel_k1_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_scaled_modified_bessel_k1_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_t_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_u_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_u_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_w_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_w_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_spherical_bessel_j0_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_spherical_bessel_j0_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_xlog1py_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_special_xlog1py_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_list_args_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_list_args_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_list_args_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_copy_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_copy_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_copy_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_copy_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sqrt_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sqrt_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sqrt_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_square_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_square_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_squeeze_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_squeeze_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_squeeze_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_squeeze_multiple_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_stack_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_std_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_std_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_std_mean_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_std_mean_unbiased_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_std_unbiased_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sub_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sub_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sum_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_sum_to_size_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_svd_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_t_copy_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_t_copy_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_t_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_t_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_t_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_take_along_dim_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_take_along_dim_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_take_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_tan_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_tan_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_tensor_split_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_tile_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_to_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_to_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_to_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_to_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_to_sparse_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_topk_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_topk_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_torch_ops_aten__efficient_attention_forward_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_torch_ops_aten__safe_softmax_default_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_trace_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_trace_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_trace_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_trapezoid_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_trapz_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_trapz_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_triangular_solve_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_tril_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_tril_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_tril_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_triu_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_triu_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_triu_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_trunc_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unbind_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unbind_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unfold_copy_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unfold_copy_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_uniform_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unique_consecutive_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unique_cuda_uint16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unravel_index_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unsafe_chunk_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unsafe_chunk_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unsafe_split_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unsqueeze_copy_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unsqueeze_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unsqueeze_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_unsqueeze_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_var_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_var_mean_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_var_mean_unbiased_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_var_mean_unbiased_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_vdot_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_vdot_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_view_as_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_view_copy_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_view_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_vsplit_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_vstack_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_where_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_where_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_where_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_where_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_comprehensive_xlogy_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_comprehensive_zero__cuda_float64, test/test_decomp.py::TestDecompCUDA::test_comprehensive_zeros_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_comprehensive_zeros_like_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_comprehensive_zeros_like_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_masked_fill_cuda, test/test_decomp.py::TestDecompCUDA::test_quick__batch_norm_with_update_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick__chunk_cat_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick__chunk_cat_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick__chunk_cat_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick__chunk_cat_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_put_accumulate_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_put_accumulate_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_put_accumulate_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick__upsample_bilinear2d_aa_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_acos_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_acos_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_acos_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_add_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_addcdiv_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_addcmul_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_addcmul_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_addmm_decomposed_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_addmv_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_addr_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_alias_copy_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_amin_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_amin_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_aminmax_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_any_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_arange_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_arange_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_as_strided_scatter_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_as_strided_scatter_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_asin_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_asin_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_asin_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_asin_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_asinh_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_asinh_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_asinh_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_atan2_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_atan2_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_atan2_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_atan_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_atanh_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_atanh_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_atanh_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_bitwise_not_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_bitwise_not_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_bitwise_not_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_bitwise_or_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_bitwise_right_shift_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_bitwise_xor_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_block_diag_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_bucketize_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_cat_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_cat_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_cauchy_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_ceil_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_clamp_max_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_clamp_min_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_clone_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_clone_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_clone_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_complex_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_conj_physical_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_conj_physical_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_conj_physical_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_constant_pad_nd_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_deg2rad_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_expand_copy_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_logaddexp_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_mvlgamma_mvlgamma_p_3_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_nansum_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_norm_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_norm_inf_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_norm_nuc_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_split_with_sizes_copy_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_std_mean_unbiased_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_std_unbiased_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_sum_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_trace_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_tril_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_vdot_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_zero__cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_cos_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_cos_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_cosh_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_count_nonzero_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_count_nonzero_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_count_nonzero_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_count_nonzero_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_cumprod_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_cumprod_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_cumsum_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_deg2rad_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_diag_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_diag_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_diag_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_diag_embed_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_diag_embed_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_diag_embed_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_diag_embed_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_diagonal_copy_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_diagonal_copy_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_diagonal_copy_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_diagonal_copy_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_diagonal_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_diagonal_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_diagonal_scatter_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_diagonal_scatter_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_digamma_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_digamma_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_div_floor_rounding_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_div_no_rounding_mode_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_div_no_rounding_mode_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_div_no_rounding_mode_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_div_no_rounding_mode_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_empty_like_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_empty_like_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_empty_like_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_empty_like_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_eq_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_erf_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_erf_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_erf_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_erfc_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_erfinv_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_erfinv_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_erfinv_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_exp_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_exp_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_expand_copy_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_expand_copy_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_expand_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_expand_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_expm1_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_exponential_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_eye_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_fft_fft2_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_fft_fft2_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_fft_fft2_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_fft_fftn_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_fft_hfft_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_fft_ifft2_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_fft_ifft2_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_fft_ifft_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_fft_ifft_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_fft_ifft_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_fft_ifft_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_fft_ifftn_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_fft_ihfft2_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_fft_ihfft_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_fft_irfft_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_fft_irfftn_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_fft_irfftn_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_fft_rfft_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_fill_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_fill_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_fill_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_fill_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_flip_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_floor_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_floor_divide_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_fmin_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_fmin_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_fmod_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_fmod_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_full_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_full_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_full_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_gcd_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_gcd_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_ge_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_geometric_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_heaviside_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_i0_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_igammac_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_index_add_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_index_add_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_index_copy_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_index_fill_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_index_select_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_index_select_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_isin_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_isnan_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_isnan_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_isnan_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_isneginf_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_item_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_lcm_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_lerp_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_lgamma_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_lgamma_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_lgamma_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_linalg_cross_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_linalg_diagonal_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_linspace_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_linspace_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_linspace_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_linspace_tensor_overload_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_log10_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_log10_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_log2_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_log_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_log_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_log_softmax_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_log_softmax_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_log_softmax_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_logaddexp2_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_logaddexp_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_logical_and_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_logical_and_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_logical_and_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_logical_not_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_logical_or_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_logical_or_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_logit_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_logit_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_logspace_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_logspace_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_logsumexp_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_logsumexp_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_logsumexp_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_logsumexp_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_lt_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_lt_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_masked_fill_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_maximum_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_meshgrid_list_of_tensors_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_meshgrid_variadic_tensors_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_minimum_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_minimum_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_mul_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_mul_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_mul_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_mv_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_mvlgamma_mvlgamma_p_3_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_mvlgamma_mvlgamma_p_3_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_mvlgamma_mvlgamma_p_3_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_nan_to_num_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_nan_to_num_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_nan_to_num_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_nansum_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_narrow_copy_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_narrow_copy_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_native_batch_norm_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_native_layer_norm_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_ne_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_ne_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_neg_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_neg_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_new_empty_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_new_empty_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_new_empty_strided_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_new_ones_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_new_ones_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_nextafter_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_hardsigmoid_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_hardswish_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_leaky_relu_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_logsigmoid_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_pad_constant_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_pad_constant_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_prelu_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_relu_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_rrelu_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_softplus_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_nn_functional_unfold_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_norm_inf_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_norm_nuc_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_normal_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_normal_in_place_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_normal_number_mean_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_ones_like_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_ones_like_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_permute_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_permute_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_polar_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_prod_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_prod_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_prod_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_prod_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_rad2deg_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_randn_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_randn_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_remainder_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_remainder_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_repeat_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_repeat_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_roll_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_roll_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_round_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_round_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_round_decimals_3_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_round_decimals_neg_3_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_round_decimals_neg_3_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_rsqrt_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_rsqrt_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_rsub_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_rsub_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_select_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_sgn_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_sgn_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_sigmoid_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_sigmoid_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_sigmoid_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_sign_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_sign_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_sign_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_signbit_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_sin_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_sin_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_sin_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_sinc_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_sinc_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_sinh_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_slice_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_slice_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_slice_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_slice_scatter_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_special_entr_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_special_entr_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_special_entr_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_special_i0e_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_special_i0e_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_special_i1_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_special_log_ndtr_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_special_log_ndtr_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_special_log_ndtr_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_special_ndtr_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_special_ndtr_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_special_zeta_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_split_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_split_list_args_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_split_list_args_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_split_with_sizes_copy_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_split_with_sizes_copy_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_split_with_sizes_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_sqrt_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_sqrt_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_squeeze_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_squeeze_multiple_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_std_cuda_float32, test/test_decomp.py::TestDecompCUDA::test_quick_std_mean_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_std_mean_unbiased_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_sub_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_sub_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_sum_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_t_copy_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_t_copy_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_t_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_t_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_t_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_take_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_take_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_tan_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_tanh_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_transpose_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_tril_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_tril_indices_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_triu_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_triu_cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_triu_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_triu_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_triu_indices_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_trunc_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_unbind_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_unfold_copy_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_unfold_cuda_complex128, test/test_decomp.py::TestDecompCUDA::test_quick_unsafe_split_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_unsafe_split_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_unsafe_split_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_unsqueeze_copy_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_unsqueeze_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_unsqueeze_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_var_unbiased_cuda_float16, test/test_decomp.py::TestDecompCUDA::test_quick_vdot_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_view_copy_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_view_copy_cuda_float64, test/test_decomp.py::TestDecompCUDA::test_quick_view_copy_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_view_cuda_bfloat16, test/test_decomp.py::TestDecompCUDA::test_quick_view_cuda_complex32, test/test_decomp.py::TestDecompCUDA::test_quick_view_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_view_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_view_cuda_uint8, test/test_decomp.py::TestDecompCUDA::test_quick_where_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_where_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_quick_xlogy_cuda_int32, test/test_decomp.py::TestDecompCUDA::test_quick_zero__cuda_complex64, test/test_decomp.py::TestDecompCUDA::test_quick_zeros_cuda_bool, test/test_decomp.py::TestDecompCUDA::test_quick_zeros_cuda_int16, test/test_decomp.py::TestDecompCUDA::test_quick_zeros_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_zeros_like_cuda_int64, test/test_decomp.py::TestDecompCUDA::test_quick_zeros_like_cuda_int8, test/test_decomp.py::TestDecompCUDA::test_uniform_cuda, test/test_decomp.py::DecompOneOffTestsCUDA::test_threshold_backward_dtype_cuda 2024-08-08T21:30:11.9284572Z 2024-08-08T21:30:11.9285512Z test_decomp.py::TestDecompCUDA::test_cat_single_input_cuda PASSED [0.2196s] [ 0%] 2024-08-08T21:30:11.9286992Z test_decomp.py::TestDecompCUDA::test_comprehensive_H_cuda_float16 PASSED [0.1228s] [ 0%] 2024-08-08T21:30:11.9288466Z test_decomp.py::TestDecompCUDA::test_comprehensive_H_cuda_float32 PASSED [0.2482s] [ 0%] 2024-08-08T21:30:11.9289916Z test_decomp.py::TestDecompCUDA::test_comprehensive_T_cuda_float32 PASSED [0.0945s] [ 0%] 2024-08-08T21:30:11.9291419Z test_decomp.py::TestDecompCUDA::test_comprehensive___getitem___cuda_float64 PASSED [3.0614s] [ 0%] 2024-08-08T21:30:11.9293832Z test_decomp.py::TestDecompCUDA::test_comprehensive___getitem___cuda_int16 PASSED [0.3645s] [ 0%] 2024-08-08T21:30:11.9295364Z test_decomp.py::TestDecompCUDA::test_comprehensive___getitem___cuda_int8 PASSED [0.3609s] [ 0%] 2024-08-08T21:30:11.9297183Z test_decomp.py::TestDecompCUDA::test_comprehensive___radd___cuda_bool PASSED [0.1879s] [ 0%] 2024-08-08T21:30:11.9298775Z test_decomp.py::TestDecompCUDA::test_comprehensive___radd___cuda_int32 PASSED [0.2065s] [ 0%] 2024-08-08T21:30:11.9300273Z test_decomp.py::TestDecompCUDA::test_comprehensive___radd___cuda_int8 PASSED [0.1887s] [ 0%] 2024-08-08T21:30:11.9301760Z test_decomp.py::TestDecompCUDA::test_comprehensive___radd___cuda_uint8 PASSED [0.1886s] [ 0%] 2024-08-08T21:30:11.9303253Z test_decomp.py::TestDecompCUDA::test_comprehensive___rand___cuda_int16 PASSED [0.1896s] [ 0%] 2024-08-08T21:30:11.9304746Z test_decomp.py::TestDecompCUDA::test_comprehensive___rmod___cuda_float32 PASSED [1.9546s] [ 1%] 2024-08-08T21:30:11.9306268Z test_decomp.py::TestDecompCUDA::test_comprehensive___rmod___cuda_float64 PASSED [1.9049s] [ 1%] 2024-08-08T21:30:11.9307803Z test_decomp.py::TestDecompCUDA::test_comprehensive___rmul___cuda_bfloat16 PASSED [0.0854s] [ 1%] 2024-08-08T21:30:11.9309375Z test_decomp.py::TestDecompCUDA::test_comprehensive___rmul___cuda_complex128 PASSED [2.9711s] [ 1%] 2024-08-08T21:30:11.9310900Z test_decomp.py::TestDecompCUDA::test_comprehensive___rmul___cuda_float16 PASSED [0.0851s] [ 1%] 2024-08-08T21:30:11.9312408Z test_decomp.py::TestDecompCUDA::test_comprehensive___rmul___cuda_uint8 PASSED [0.1907s] [ 1%] 2024-08-08T21:30:11.9313896Z test_decomp.py::TestDecompCUDA::test_comprehensive___ror___cuda_int16 PASSED [0.1893s] [ 1%] 2024-08-08T21:30:11.9315351Z test_decomp.py::TestDecompCUDA::test_comprehensive___ror___cuda_uint8 PASSED [0.1878s] [ 1%] 2024-08-08T21:30:11.9316856Z test_decomp.py::TestDecompCUDA::test_comprehensive___rpow___cuda_float32 PASSED [3.9762s] [ 1%] 2024-08-08T21:30:11.9318360Z test_decomp.py::TestDecompCUDA::test_comprehensive___rpow___cuda_int8 PASSED [0.1877s] [ 1%] 2024-08-08T21:30:11.9319955Z test_decomp.py::TestDecompCUDA::test_comprehensive___rsub___cuda_int32 PASSED [0.3211s] [ 1%] 2024-08-08T21:30:11.9321653Z test_decomp.py::TestDecompCUDA::test_comprehensive__batch_norm_with_update_cuda_float32 PASSED [4.5180s] [ 1%] 2024-08-08T21:30:11.9323382Z test_decomp.py::TestDecompCUDA::test_comprehensive__native_batch_norm_legit_cuda_float64 PASSED [4.9011s] [ 2%] 2024-08-08T21:30:11.9325107Z test_decomp.py::TestDecompCUDA::test_comprehensive__segment_reduce_lengths_cuda_float32 PASSED [0.1722s] [ 2%] 2024-08-08T21:30:11.9326827Z test_decomp.py::TestDecompCUDA::test_comprehensive__segment_reduce_lengths_cuda_float64 PASSED [0.1618s] [ 2%] 2024-08-08T21:30:11.9328515Z test_decomp.py::TestDecompCUDA::test_comprehensive__softmax_backward_data_cuda_bfloat16 PASSED [0.0607s] [ 2%] 2024-08-08T21:30:11.9330241Z test_decomp.py::TestDecompCUDA::test_comprehensive__softmax_backward_data_cuda_float64 PASSED [0.8698s] [ 2%] 2024-08-08T21:30:11.9331952Z test_decomp.py::TestDecompCUDA::test_comprehensive__unsafe_masked_index_cuda_int8 PASSED [2.7476s] [ 2%] 2024-08-08T21:30:11.9333696Z test_decomp.py::TestDecompCUDA::test_comprehensive__unsafe_masked_index_put_accumulate_cuda_uint8 PASSED [2.8957s] [ 2%] 2024-08-08T21:30:11.9335328Z test_decomp.py::TestDecompCUDA::test_comprehensive_abs_cuda_bool PASSED [0.0387s] [ 2%] 2024-08-08T21:30:11.9336799Z test_decomp.py::TestDecompCUDA::test_comprehensive_add_cuda_float16 PASSED [0.0606s] [ 2%] 2024-08-08T21:30:11.9338298Z test_decomp.py::TestDecompCUDA::test_comprehensive_addbmm_cuda_float32 PASSED [1.1403s] [ 2%] 2024-08-08T21:30:11.9339804Z test_decomp.py::TestDecompCUDA::test_comprehensive_addcdiv_cuda_float16 PASSED [0.2901s] [ 2%] 2024-08-08T21:30:11.9341394Z test_decomp.py::TestDecompCUDA::test_comprehensive_addcdiv_cuda_float32 PASSED [2.2278s] [ 2%] 2024-08-08T21:30:11.9342976Z test_decomp.py::TestDecompCUDA::test_comprehensive_addcmul_cuda_float64 PASSED [1.6371s] [ 3%] 2024-08-08T21:30:11.9344489Z test_decomp.py::TestDecompCUDA::test_comprehensive_addcmul_cuda_int16 PASSED [0.3741s] [ 3%] 2024-08-08T21:30:11.9345962Z test_decomp.py::TestDecompCUDA::test_comprehensive_addcmul_cuda_int32 PASSED [0.3754s] [ 3%] 2024-08-08T21:30:11.9347482Z test_decomp.py::TestDecompCUDA::test_comprehensive_addmm_cuda_complex64 PASSED [0.9197s] [ 3%] 2024-08-08T21:30:11.9349100Z test_decomp.py::TestDecompCUDA::test_comprehensive_addmm_decomposed_cuda_complex128 PASSED [0.6647s] [ 3%] 2024-08-08T21:30:11.9350746Z test_decomp.py::TestDecompCUDA::test_comprehensive_addmm_decomposed_cuda_float64 PASSED [0.3337s] [ 3%] 2024-08-08T21:30:11.9352334Z test_decomp.py::TestDecompCUDA::test_comprehensive_addmv_cuda_bfloat16 PASSED [0.1661s] [ 3%] 2024-08-08T21:30:11.9353919Z test_decomp.py::TestDecompCUDA::test_comprehensive_addmv_cuda_float32 PASSED [1.1082s] [ 3%] 2024-08-08T21:30:11.9355445Z test_decomp.py::TestDecompCUDA::test_comprehensive_addr_cuda_complex128 PASSED [1.3887s] [ 3%] 2024-08-08T21:30:11.9356912Z test_decomp.py::TestDecompCUDA::test_comprehensive_addr_cuda_int8 PASSED [0.1361s] [ 3%] 2024-08-08T21:30:11.9358811Z test_decomp.py::TestDecompCUDA::test_comprehensive_alias_copy_cuda_float64 PASSED [0.0283s] [ 3%] 2024-08-08T21:30:11.9360440Z test_decomp.py::TestDecompCUDA::test_comprehensive_all_cuda_bfloat16 PASSED [0.3188s] [ 3%] 2024-08-08T21:30:11.9361916Z test_decomp.py::TestDecompCUDA::test_comprehensive_all_cuda_int32 PASSED [0.3510s] [ 3%] 2024-08-08T21:30:11.9363377Z test_decomp.py::TestDecompCUDA::test_comprehensive_amax_cuda_float64 PASSED [0.7854s] [ 4%] 2024-08-08T21:30:11.9364959Z test_decomp.py::TestDecompCUDA::test_comprehensive_amin_cuda_int16 PASSED [0.0675s] [ 4%] 2024-08-08T21:30:11.9366496Z test_decomp.py::TestDecompCUDA::test_comprehensive_amin_cuda_int64 PASSED [0.0678s] [ 4%] 2024-08-08T21:30:11.9367939Z test_decomp.py::TestDecompCUDA::test_comprehensive_angle_cuda_bool PASSED [0.0058s] [ 4%] 2024-08-08T21:30:11.9369417Z test_decomp.py::TestDecompCUDA::test_comprehensive_angle_cuda_int32 PASSED [0.0053s] [ 4%] 2024-08-08T21:30:11.9370905Z test_decomp.py::TestDecompCUDA::test_comprehensive_any_cuda_bfloat16 PASSED [0.1354s] [ 4%] 2024-08-08T21:30:11.9372400Z test_decomp.py::TestDecompCUDA::test_comprehensive_any_cuda_complex64 PASSED [0.2539s] [ 4%] 2024-08-08T21:30:11.9373861Z test_decomp.py::TestDecompCUDA::test_comprehensive_any_cuda_int16 PASSED [0.1520s] [ 4%] 2024-08-08T21:30:11.9375346Z test_decomp.py::TestDecompCUDA::test_comprehensive_arange_cuda_int32 PASSED [0.0432s] [ 4%] 2024-08-08T21:30:11.9376835Z test_decomp.py::TestDecompCUDA::test_comprehensive_arange_cuda_uint8 PASSED [0.0406s] [ 4%] 2024-08-08T21:30:11.9378324Z test_decomp.py::TestDecompCUDA::test_comprehensive_argmax_cuda_int32 PASSED [0.0192s] [ 4%] 2024-08-08T21:30:11.9379803Z test_decomp.py::TestDecompCUDA::test_comprehensive_argmin_cuda_float32 PASSED [0.0188s] [ 4%] 2024-08-08T21:30:11.9381334Z test_decomp.py::TestDecompCUDA::test_comprehensive_argsort_cuda_bfloat16 PASSED [0.0624s] [ 5%] 2024-08-08T21:30:11.9382855Z test_decomp.py::TestDecompCUDA::test_comprehensive_argsort_cuda_int16 PASSED [0.0337s] [ 5%] 2024-08-08T21:30:11.9384379Z test_decomp.py::TestDecompCUDA::test_comprehensive_argwhere_cuda_complex64 PASSED [0.0101s] [ 5%] 2024-08-08T21:30:11.9386019Z test_decomp.py::TestDecompCUDA::test_comprehensive_argwhere_cuda_int64 PASSED [0.0099s] [ 5%] 2024-08-08T21:30:11.9387679Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_copy_cuda_float32 PASSED [0.1649s] [ 5%] 2024-08-08T21:30:11.9389291Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_copy_cuda_int8 PASSED [0.0310s] [ 5%] 2024-08-08T21:30:11.9390861Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_cuda_complex32 PASSED [0.0079s] [ 5%] 2024-08-08T21:30:11.9392462Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_cuda_complex64 PASSED [0.1708s] [ 5%] 2024-08-08T21:30:11.9394024Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_cuda_int32 PASSED [0.0079s] [ 5%] 2024-08-08T21:30:11.9395687Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_bfloat16 PASSED [0.0157s] [ 5%] 2024-08-08T21:30:11.9397447Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_complex128 PASSED [0.0575s] [ 5%] 2024-08-08T21:30:11.9399195Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_float32 PASSED [0.0389s] [ 5%] 2024-08-08T21:30:11.9400919Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_int32 PASSED [0.0070s] [ 6%] 2024-08-08T21:30:11.9402600Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_int64 PASSED [0.0066s] [ 6%] 2024-08-08T21:30:11.9404303Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_partial_views_cuda_int8 PASSED [0.0065s] [ 6%] 2024-08-08T21:30:11.9406334Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_scatter_cuda_complex128 SKIPPED [0.0003s] (Expected: new_empty_strided is not comparable) [ 6%] 2024-08-08T21:30:11.9408766Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_scatter_cuda_float32 SKIPPED [0.0002s] (Expected: new_empty_strided is not comparable) [ 6%] 2024-08-08T21:30:11.9411062Z test_decomp.py::TestDecompCUDA::test_comprehensive_as_strided_scatter_cuda_uint8 SKIPPED [0.0002s] (Expected: new_empty_strided is not comparable) [ 6%] 2024-08-08T21:30:11.9412873Z test_decomp.py::TestDecompCUDA::test_comprehensive_asin_cuda_bool PASSED [0.0453s] [ 6%] 2024-08-08T21:30:11.9414346Z test_decomp.py::TestDecompCUDA::test_comprehensive_asin_cuda_float16 PASSED [0.0135s] [ 6%] 2024-08-08T21:30:11.9415836Z test_decomp.py::TestDecompCUDA::test_comprehensive_asin_cuda_float64 PASSED [0.3827s] [ 6%] 2024-08-08T21:30:11.9417321Z test_decomp.py::TestDecompCUDA::test_comprehensive_atan2_cuda_bfloat16 PASSED [0.1771s] [ 6%] 2024-08-08T21:30:11.9418825Z test_decomp.py::TestDecompCUDA::test_comprehensive_atan2_cuda_int64 PASSED [0.2537s] [ 6%] 2024-08-08T21:30:11.9420325Z test_decomp.py::TestDecompCUDA::test_comprehensive_atan_cuda_bfloat16 PASSED [0.0126s] [ 6%] 2024-08-08T21:30:11.9421827Z test_decomp.py::TestDecompCUDA::test_comprehensive_atan_cuda_float32 PASSED [0.2845s] [ 7%] 2024-08-08T21:30:11.9423292Z test_decomp.py::TestDecompCUDA::test_comprehensive_atan_cuda_float64 PASSED [0.2890s] [ 7%] 2024-08-08T21:30:11.9424769Z test_decomp.py::TestDecompCUDA::test_comprehensive_atanh_cuda_bool PASSED [0.0410s] [ 7%] 2024-08-08T21:30:11.9426248Z test_decomp.py::TestDecompCUDA::test_comprehensive_atanh_cuda_int16 PASSED [0.0398s] [ 7%] 2024-08-08T21:30:11.9427703Z test_decomp.py::TestDecompCUDA::test_comprehensive_atanh_cuda_uint8 PASSED [0.0394s] [ 7%] 2024-08-08T21:30:11.9429273Z test_decomp.py::TestDecompCUDA::test_comprehensive_atleast_2d_cuda_int64 PASSED [0.0144s] [ 7%] 2024-08-08T21:30:11.9430898Z test_decomp.py::TestDecompCUDA::test_comprehensive_atleast_3d_cuda_int32 PASSED [0.0260s] [ 7%] 2024-08-08T21:30:11.9432459Z test_decomp.py::TestDecompCUDA::test_comprehensive_baddbmm_cuda_complex128 PASSED [8.4870s] [ 7%] 2024-08-08T21:30:11.9434008Z test_decomp.py::TestDecompCUDA::test_comprehensive_baddbmm_cuda_complex64 PASSED [8.5283s] [ 7%] 2024-08-08T21:30:11.9435561Z test_decomp.py::TestDecompCUDA::test_comprehensive_bernoulli_cuda_float64 PASSED [0.0424s] [ 7%] 2024-08-08T21:30:11.9437087Z test_decomp.py::TestDecompCUDA::test_comprehensive_bfloat16_cuda_bool PASSED [0.0412s] [ 7%] 2024-08-08T21:30:11.9438611Z test_decomp.py::TestDecompCUDA::test_comprehensive_bfloat16_cuda_float32 PASSED [0.0840s] [ 7%] 2024-08-08T21:30:11.9440123Z test_decomp.py::TestDecompCUDA::test_comprehensive_bincount_cuda_int64 PASSED [0.0230s] [ 7%] 2024-08-08T21:30:11.9441658Z test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_and_cuda_int16 PASSED [0.1875s] [ 8%] 2024-08-08T21:30:11.9443251Z test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_left_shift_cuda_int32 PASSED [0.1984s] [ 8%] 2024-08-08T21:30:11.9444867Z test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_left_shift_cuda_int8 PASSED [0.1864s] [ 8%] 2024-08-08T21:30:11.9446441Z test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_not_cuda_bool PASSED [0.0112s] [ 8%] 2024-08-08T21:30:11.9448039Z test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_right_shift_cuda_int8 PASSED [0.1867s] [ 8%] 2024-08-08T21:30:11.9449618Z test_decomp.py::TestDecompCUDA::test_comprehensive_bitwise_xor_cuda_int8 PASSED [0.1878s] [ 8%] 2024-08-08T21:30:11.9451137Z test_decomp.py::TestDecompCUDA::test_comprehensive_block_diag_cuda_int32 PASSED [0.2343s] [ 8%] 2024-08-08T21:30:11.9452678Z test_decomp.py::TestDecompCUDA::test_comprehensive_block_diag_cuda_uint8 PASSED [0.2344s] [ 8%] 2024-08-08T21:30:11.9454325Z test_decomp.py::TestDecompCUDA::test_comprehensive_bool_cuda_complex32 PASSED [0.0595s] [ 8%] 2024-08-08T21:30:11.9455875Z test_decomp.py::TestDecompCUDA::test_comprehensive_bool_cuda_int32 PASSED [0.0303s] [ 8%] 2024-08-08T21:30:11.9457412Z test_decomp.py::TestDecompCUDA::test_comprehensive_broadcast_shapes_cuda_float32 PASSED [0.0094s] [ 8%] 2024-08-08T21:30:11.9459477Z test_decomp.py::TestDecompCUDA::test_comprehensive_broadcast_tensors_cuda_float32 PASSED [0.0683s] [ 8%] 2024-08-08T21:30:11.9461111Z test_decomp.py::TestDecompCUDA::test_comprehensive_broadcast_to_cuda_int32 PASSED [0.0240s] [ 9%] 2024-08-08T21:30:11.9462628Z test_decomp.py::TestDecompCUDA::test_comprehensive_byte_cuda_float32 PASSED [0.0415s] [ 9%] 2024-08-08T21:30:11.9464091Z test_decomp.py::TestDecompCUDA::test_comprehensive_byte_cuda_int8 PASSED [0.0302s] [ 9%] 2024-08-08T21:30:11.9465613Z test_decomp.py::TestDecompCUDA::test_comprehensive_cdouble_cuda_complex64 PASSED [0.1278s] [ 9%] 2024-08-08T21:30:11.9467126Z test_decomp.py::TestDecompCUDA::test_comprehensive_ceil_cuda_int16 PASSED [0.0402s] [ 9%] 2024-08-08T21:30:11.9468590Z test_decomp.py::TestDecompCUDA::test_comprehensive_cfloat_cuda_float16 PASSED [0.0475s] [ 9%] 2024-08-08T21:30:11.9470089Z test_decomp.py::TestDecompCUDA::test_comprehensive_cfloat_cuda_int16 PASSED [0.0502s] [ 9%] 2024-08-08T21:30:11.9471581Z test_decomp.py::TestDecompCUDA::test_comprehensive_cfloat_cuda_int32 PASSED [0.0496s] [ 9%] 2024-08-08T21:30:11.9473054Z test_decomp.py::TestDecompCUDA::test_comprehensive_chalf_cuda_int16 PASSED [0.3100s] [ 9%] 2024-08-08T21:30:11.9474613Z test_decomp.py::TestDecompCUDA::test_comprehensive_chalf_cuda_int64 PASSED [0.0508s] [ 9%] 2024-08-08T21:30:11.9476207Z test_decomp.py::TestDecompCUDA::test_comprehensive_char_cuda_complex128 PASSED [0.0576s] [ 9%] 2024-08-08T21:30:11.9477737Z test_decomp.py::TestDecompCUDA::test_comprehensive_char_cuda_complex64 PASSED [0.0578s] [ 9%] 2024-08-08T21:30:11.9479208Z test_decomp.py::TestDecompCUDA::test_comprehensive_char_cuda_int64 PASSED [0.0303s] [ 10%] 2024-08-08T21:30:11.9480673Z test_decomp.py::TestDecompCUDA::test_comprehensive_char_cuda_uint8 PASSED [0.0302s] [ 10%] 2024-08-08T21:30:11.9482244Z test_decomp.py::TestDecompCUDA::test_comprehensive_cholesky_inverse_cuda_complex64 PASSED [1.6533s] [ 10%] 2024-08-08T21:30:11.9483911Z test_decomp.py::TestDecompCUDA::test_comprehensive_cholesky_solve_cuda_complex128 PASSED [1.5136s] [ 10%] 2024-08-08T21:30:11.9485537Z test_decomp.py::TestDecompCUDA::test_comprehensive_cholesky_solve_cuda_float32 PASSED [0.8432s] [ 10%] 2024-08-08T21:30:11.9487110Z test_decomp.py::TestDecompCUDA::test_comprehensive_chunk_cuda_float32 PASSED [0.6817s] [ 10%] 2024-08-08T21:30:11.9488630Z test_decomp.py::TestDecompCUDA::test_comprehensive_clamp_max_cuda_int16 PASSED [1.1974s] [ 10%] 2024-08-08T21:30:11.9490168Z test_decomp.py::TestDecompCUDA::test_comprehensive_clamp_min_cuda_float64 PASSED [3.3224s] [ 10%] 2024-08-08T21:30:11.9491685Z test_decomp.py::TestDecompCUDA::test_comprehensive_clamp_min_cuda_int64 PASSED [1.1836s] [ 10%] 2024-08-08T21:30:11.9493185Z test_decomp.py::TestDecompCUDA::test_comprehensive_clone_cuda_bool PASSED [0.0362s] [ 10%] 2024-08-08T21:30:11.9494674Z test_decomp.py::TestDecompCUDA::test_comprehensive_clone_cuda_float32 PASSED [0.0488s] [ 10%] 2024-08-08T21:30:11.9496153Z test_decomp.py::TestDecompCUDA::test_comprehensive_clone_cuda_uint8 PASSED [0.0362s] [ 10%] 2024-08-08T21:30:11.9497782Z test_decomp.py::TestDecompCUDA::test_comprehensive_column_stack_cuda_bfloat16 PASSED [0.0237s] [ 10%] 2024-08-08T21:30:11.9499433Z test_decomp.py::TestDecompCUDA::test_comprehensive_column_stack_cuda_bool PASSED [0.0407s] [ 11%] 2024-08-08T21:30:11.9501024Z test_decomp.py::TestDecompCUDA::test_comprehensive_combinations_cuda_bfloat16 PASSED [1.1235s] [ 11%] 2024-08-08T21:30:11.9502601Z test_decomp.py::TestDecompCUDA::test_comprehensive_combinations_cuda_int64 PASSED [1.7348s] [ 11%] 2024-08-08T21:30:11.9504126Z test_decomp.py::TestDecompCUDA::test_comprehensive_conj_cuda_int64 PASSED [0.0063s] [ 11%] 2024-08-08T21:30:11.9505587Z test_decomp.py::TestDecompCUDA::test_comprehensive_conj_cuda_int8 PASSED [0.0064s] [ 11%] 2024-08-08T21:30:11.9507096Z test_decomp.py::TestDecompCUDA::test_comprehensive_conj_physical_cuda_int8 PASSED [0.0053s] [ 11%] 2024-08-08T21:30:11.9508739Z test_decomp.py::TestDecompCUDA::test_comprehensive_corrcoef_cuda_float32 PASSED [1.4811s] [ 11%] 2024-08-08T21:30:11.9510288Z test_decomp.py::TestDecompCUDA::test_comprehensive_corrcoef_cuda_float64 PASSED [1.3951s] [ 11%] 2024-08-08T21:30:11.9511816Z test_decomp.py::TestDecompCUDA::test_comprehensive_corrcoef_cuda_uint8 PASSED [0.5644s] [ 11%] 2024-08-08T21:30:11.9513281Z test_decomp.py::TestDecompCUDA::test_comprehensive_cos_cuda_bool PASSED [0.0146s] [ 11%] 2024-08-08T21:30:11.9514747Z test_decomp.py::TestDecompCUDA::test_comprehensive_cos_cuda_float16 PASSED [0.0223s] [ 11%] 2024-08-08T21:30:11.9516236Z test_decomp.py::TestDecompCUDA::test_comprehensive_cosh_cuda_float16 PASSED [0.0209s] [ 11%] 2024-08-08T21:30:11.9517780Z test_decomp.py::TestDecompCUDA::test_comprehensive_cosh_cuda_int64 PASSED [0.0128s] [ 12%] 2024-08-08T21:30:11.9519359Z test_decomp.py::TestDecompCUDA::test_comprehensive_count_nonzero_cuda_float64 PASSED [0.2503s] [ 12%] 2024-08-08T21:30:11.9520949Z test_decomp.py::TestDecompCUDA::test_comprehensive_count_nonzero_cuda_int8 PASSED [0.1684s] [ 12%] 2024-08-08T21:30:11.9522497Z test_decomp.py::TestDecompCUDA::test_comprehensive_cov_cuda_complex128 PASSED [13.9051s] [ 12%] 2024-08-08T21:30:11.9523983Z test_decomp.py::TestDecompCUDA::test_comprehensive_cov_cuda_float16 PASSED [2.3526s] [ 12%] 2024-08-08T21:30:11.9525453Z test_decomp.py::TestDecompCUDA::test_comprehensive_cov_cuda_int32 PASSED [3.9029s] [ 12%] 2024-08-08T21:30:11.9526900Z test_decomp.py::TestDecompCUDA::test_comprehensive_cov_cuda_uint8 PASSED [3.8066s] [ 12%] 2024-08-08T21:30:11.9528369Z test_decomp.py::TestDecompCUDA::test_comprehensive_cross_cuda_int64 PASSED [0.3081s] [ 12%] 2024-08-08T21:30:11.9529857Z test_decomp.py::TestDecompCUDA::test_comprehensive_cummax_cuda_float16 PASSED [0.0247s] [ 12%] 2024-08-08T21:30:11.9531391Z test_decomp.py::TestDecompCUDA::test_comprehensive_cummax_cuda_float64 PASSED [0.0603s] [ 12%] 2024-08-08T21:30:11.9532907Z test_decomp.py::TestDecompCUDA::test_comprehensive_cummin_cuda_float64 PASSED [0.0569s] [ 12%] 2024-08-08T21:30:11.9534413Z test_decomp.py::TestDecompCUDA::test_comprehensive_cummin_cuda_int16 PASSED [0.0068s] [ 12%] 2024-08-08T21:30:11.9535939Z test_decomp.py::TestDecompCUDA::test_comprehensive_cumprod_cuda_complex128 PASSED [48.7496s] [ 13%] 2024-08-08T21:30:11.9537510Z test_decomp.py::TestDecompCUDA::test_comprehensive_cumprod_cuda_float32 PASSED [25.0537s] [ 13%] 2024-08-08T21:30:11.9539056Z test_decomp.py::TestDecompCUDA::test_comprehensive_cumprod_cuda_float64 PASSED [23.9032s] [ 13%] 2024-08-08T21:30:11.9540565Z test_decomp.py::TestDecompCUDA::test_comprehensive_cumsum_cuda_int64 PASSED [0.4588s] [ 13%] 2024-08-08T21:30:11.9542222Z test_decomp.py::TestDecompCUDA::test_comprehensive_cumulative_trapezoid_cuda_int32 PASSED [0.8468s] [ 13%] 2024-08-08T21:30:11.9544135Z test_decomp.py::TestDecompCUDA::test_comprehensive_diag_cuda_float32 SKIPPED [0.0029s] (diag in torch.float32 not supported) [ 13%] 2024-08-08T21:30:11.9545902Z test_decomp.py::TestDecompCUDA::test_comprehensive_diagflat_cuda_bfloat16 PASSED [0.1551s] [ 13%] 2024-08-08T21:30:11.9547444Z test_decomp.py::TestDecompCUDA::test_comprehensive_diagflat_cuda_float64 PASSED [0.3943s] [ 13%] 2024-08-08T21:30:11.9549014Z test_decomp.py::TestDecompCUDA::test_comprehensive_diagonal_copy_cuda_int64 PASSED [0.2282s] [ 13%] 2024-08-08T21:30:11.9550586Z test_decomp.py::TestDecompCUDA::test_comprehensive_diagonal_cuda_float32 PASSED [0.6982s] [ 13%] 2024-08-08T21:30:11.9552097Z test_decomp.py::TestDecompCUDA::test_comprehensive_diff_cuda_bool PASSED [3.7866s] [ 13%] 2024-08-08T21:30:11.9553637Z test_decomp.py::TestDecompCUDA::test_comprehensive_diff_cuda_complex64 PASSED [143.1877s] [ 13%] 2024-08-08T21:30:11.9555152Z test_decomp.py::TestDecompCUDA::test_comprehensive_diff_cuda_int16 PASSED [3.6097s] [ 14%] 2024-08-08T21:30:11.9556640Z test_decomp.py::TestDecompCUDA::test_comprehensive_digamma_cuda_int32 PASSED [0.6229s] [ 14%] 2024-08-08T21:30:11.9558221Z test_decomp.py::TestDecompCUDA::test_comprehensive_div_floor_rounding_cuda_bfloat16 PASSED [0.1499s] [ 14%] 2024-08-08T21:30:11.9560427Z test_decomp.py::TestDecompCUDA::test_comprehensive_div_floor_rounding_cuda_float32 PASSED [1.0837s] [ 14%] 2024-08-08T21:30:11.9562120Z test_decomp.py::TestDecompCUDA::test_comprehensive_div_no_rounding_mode_cuda_complex64 PASSED [5.1009s] [ 14%] 2024-08-08T21:30:11.9563912Z test_decomp.py::TestDecompCUDA::test_comprehensive_div_no_rounding_mode_cuda_int64 PASSED [0.3006s] [ 14%] 2024-08-08T21:30:11.9565628Z test_decomp.py::TestDecompCUDA::test_comprehensive_div_trunc_rounding_cuda_int16 PASSED [0.1956s] [ 14%] 2024-08-08T21:30:11.9567181Z test_decomp.py::TestDecompCUDA::test_comprehensive_dot_cuda_bfloat16 PASSED [0.0156s] [ 14%] 2024-08-08T21:30:11.9568683Z test_decomp.py::TestDecompCUDA::test_comprehensive_dstack_cuda_float32 PASSED [0.3404s] [ 14%] 2024-08-08T21:30:11.9570186Z test_decomp.py::TestDecompCUDA::test_comprehensive_dstack_cuda_int16 PASSED [0.0739s] [ 14%] 2024-08-08T21:30:11.9571647Z test_decomp.py::TestDecompCUDA::test_comprehensive_dstack_cuda_int64 PASSED [0.0746s] [ 14%] 2024-08-08T21:30:11.9573430Z test_decomp.py::TestDecompCUDA::test_comprehensive_empty_cuda_float64 SKIPPED [0.0027s] (empty in torch.float64 not supported) [ 14%] 2024-08-08T21:30:11.9575435Z test_decomp.py::TestDecompCUDA::test_comprehensive_empty_cuda_int64 SKIPPED [0.0026s] (empty in torch.int64 not supported) [ 14%] 2024-08-08T21:30:11.9577502Z test_decomp.py::TestDecompCUDA::test_comprehensive_empty_like_cuda_bfloat16 SKIPPED [0.0027s] (empty_like in torch.bfloat16 not supported) [ 15%] 2024-08-08T21:30:11.9579597Z test_decomp.py::TestDecompCUDA::test_comprehensive_empty_like_cuda_int32 SKIPPED [0.0029s] (empty_like in torch.int32 not supported) [ 15%] 2024-08-08T21:30:11.9581429Z test_decomp.py::TestDecompCUDA::test_comprehensive_empty_permuted_cuda_bfloat16 PASSED [0.0276s] [ 15%] 2024-08-08T21:30:11.9583069Z test_decomp.py::TestDecompCUDA::test_comprehensive_empty_permuted_cuda_complex64 PASSED [0.0274s] [ 15%] 2024-08-08T21:30:11.9584676Z test_decomp.py::TestDecompCUDA::test_comprehensive_empty_permuted_cuda_float32 PASSED [0.0280s] [ 15%] 2024-08-08T21:30:11.9586276Z test_decomp.py::TestDecompCUDA::test_comprehensive_empty_permuted_cuda_int8 PASSED [0.0275s] [ 15%] 2024-08-08T21:30:11.9588321Z test_decomp.py::TestDecompCUDA::test_comprehensive_empty_strided_cuda_complex128 SKIPPED [0.0027s] (empty_strided in torch.complex128 not supported) [ 15%] 2024-08-08T21:30:11.9590227Z test_decomp.py::TestDecompCUDA::test_comprehensive_eq_cuda_int32 PASSED [0.2012s] [ 15%] 2024-08-08T21:30:11.9591703Z test_decomp.py::TestDecompCUDA::test_comprehensive_equal_cuda_complex128 PASSED [0.0108s] [ 15%] 2024-08-08T21:30:11.9593244Z test_decomp.py::TestDecompCUDA::test_comprehensive_equal_cuda_complex64 PASSED [0.0110s] [ 15%] 2024-08-08T21:30:11.9594758Z test_decomp.py::TestDecompCUDA::test_comprehensive_erfinv_cuda_int64 PASSED [0.3240s] [ 15%] 2024-08-08T21:30:11.9596227Z test_decomp.py::TestDecompCUDA::test_comprehensive_exp_cuda_int64 PASSED [0.0137s] [ 15%] 2024-08-08T21:30:11.9597670Z test_decomp.py::TestDecompCUDA::test_comprehensive_exp_cuda_uint8 PASSED [0.0134s] [ 16%] 2024-08-08T21:30:11.9599159Z test_decomp.py::TestDecompCUDA::test_comprehensive_expand_as_cuda_bool PASSED [0.0120s] [ 16%] 2024-08-08T21:30:11.9600699Z test_decomp.py::TestDecompCUDA::test_comprehensive_expand_copy_cuda_int64 PASSED [0.1783s] [ 16%] 2024-08-08T21:30:11.9602191Z test_decomp.py::TestDecompCUDA::test_comprehensive_expand_cuda_bool PASSED [0.0340s] [ 16%] 2024-08-08T21:30:11.9603702Z test_decomp.py::TestDecompCUDA::test_comprehensive_expand_cuda_complex64 PASSED [0.3940s] [ 16%] 2024-08-08T21:30:11.9605222Z test_decomp.py::TestDecompCUDA::test_comprehensive_expand_cuda_float32 PASSED [0.2354s] [ 16%] 2024-08-08T21:30:11.9606739Z test_decomp.py::TestDecompCUDA::test_comprehensive_expm1_cuda_bfloat16 PASSED [0.0106s] [ 16%] 2024-08-08T21:30:11.9608297Z test_decomp.py::TestDecompCUDA::test_comprehensive_eye_cuda_float32 PASSED [4.6100s] [ 16%] 2024-08-08T21:30:11.9609881Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_fft2_cuda_complex128 PASSED [1.2738s] [ 16%] 2024-08-08T21:30:11.9611415Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_fft_cuda_int16 PASSED [0.2634s] [ 16%] 2024-08-08T21:30:11.9612900Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_fft_cuda_int64 PASSED [0.2512s] [ 16%] 2024-08-08T21:30:11.9614408Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_hfft2_cuda_float32 PASSED [3.4170s] [ 16%] 2024-08-08T21:30:11.9615949Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_hfft2_cuda_float64 PASSED [3.6633s] [ 17%] 2024-08-08T21:30:11.9617472Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_hfft_cuda_int64 PASSED [0.6317s] [ 17%] 2024-08-08T21:30:11.9619009Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifft2_cuda_complex64 PASSED [1.1352s] [ 17%] 2024-08-08T21:30:11.9620556Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifft_cuda_int64 PASSED [0.2596s] [ 17%] 2024-08-08T21:30:11.9622107Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftn_cuda_complex128 PASSED [1.1449s] [ 17%] 2024-08-08T21:30:11.9623693Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftn_cuda_complex64 PASSED [1.1390s] [ 17%] 2024-08-08T21:30:11.9625232Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftn_cuda_float32 PASSED [1.2935s] [ 17%] 2024-08-08T21:30:11.9626779Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftshift_cuda_bool PASSED [0.2619s] [ 17%] 2024-08-08T21:30:11.9628369Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftshift_cuda_float32 PASSED [0.7218s] [ 17%] 2024-08-08T21:30:11.9629951Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftshift_cuda_int16 PASSED [0.2631s] [ 17%] 2024-08-08T21:30:11.9631564Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ifftshift_cuda_uint8 PASSED [0.2634s] [ 17%] 2024-08-08T21:30:11.9633163Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ihfft2_cuda_int16 PASSED [0.4205s] [ 17%] 2024-08-08T21:30:11.9634707Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ihfft_cuda_float32 PASSED [1.0287s] [ 17%] 2024-08-08T21:30:11.9636213Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ihfft_cuda_int8 PASSED [0.2497s] [ 18%] 2024-08-08T21:30:11.9637736Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_ihfftn_cuda_int16 PASSED [0.4410s] [ 18%] 2024-08-08T21:30:11.9639264Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft2_cuda_bool PASSED [0.2647s] [ 18%] 2024-08-08T21:30:11.9640802Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft2_cuda_uint8 PASSED [0.3098s] [ 18%] 2024-08-08T21:30:11.9642333Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft_cuda_float32 PASSED [1.5462s] [ 18%] 2024-08-08T21:30:11.9643870Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft_cuda_int32 PASSED [0.2289s] [ 18%] 2024-08-08T21:30:11.9645380Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfft_cuda_int64 PASSED [0.2273s] [ 18%] 2024-08-08T21:30:11.9646886Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_irfftn_cuda_bool PASSED [0.2865s] [ 18%] 2024-08-08T21:30:11.9648370Z test_decomp.py::TestDecompCUDA::test_comprehensive_fft_rfftn_cuda_int16 PASSED [0.4431s] [ 18%] 2024-08-08T21:30:11.9649896Z test_decomp.py::TestDecompCUDA::test_comprehensive_flatten_cuda_bfloat16 PASSED [0.0179s] [ 18%] 2024-08-08T21:30:11.9651478Z test_decomp.py::TestDecompCUDA::test_comprehensive_flatten_cuda_float32 PASSED [0.1075s] [ 18%] 2024-08-08T21:30:11.9653037Z test_decomp.py::TestDecompCUDA::test_comprehensive_flatten_cuda_int16 PASSED [0.0444s] [ 18%] 2024-08-08T21:30:11.9654574Z test_decomp.py::TestDecompCUDA::test_comprehensive_flip_cuda_bool PASSED [0.1603s] [ 19%] 2024-08-08T21:30:11.9656067Z test_decomp.py::TestDecompCUDA::test_comprehensive_flip_cuda_complex64 PASSED [0.6951s] [ 19%] 2024-08-08T21:30:11.9657569Z test_decomp.py::TestDecompCUDA::test_comprehensive_flip_cuda_float32 PASSED [0.4145s] [ 19%] 2024-08-08T21:30:11.9659485Z test_decomp.py::TestDecompCUDA::test_comprehensive_fliplr_cuda_complex128 PASSED [0.1446s] [ 19%] 2024-08-08T21:30:11.9661035Z test_decomp.py::TestDecompCUDA::test_comprehensive_fliplr_cuda_float32 PASSED [0.0870s] [ 19%] 2024-08-08T21:30:11.9662533Z test_decomp.py::TestDecompCUDA::test_comprehensive_fliplr_cuda_int32 PASSED [0.0362s] [ 19%] 2024-08-08T21:30:11.9664030Z test_decomp.py::TestDecompCUDA::test_comprehensive_flipud_cuda_int8 PASSED [0.0367s] [ 19%] 2024-08-08T21:30:11.9665492Z test_decomp.py::TestDecompCUDA::test_comprehensive_float_cuda_int16 PASSED [0.0479s] [ 19%] 2024-08-08T21:30:11.9666963Z test_decomp.py::TestDecompCUDA::test_comprehensive_float_cuda_int32 PASSED [0.0483s] [ 19%] 2024-08-08T21:30:11.9668482Z test_decomp.py::TestDecompCUDA::test_comprehensive_float_power_cuda_int16 PASSED [0.6304s] [ 19%] 2024-08-08T21:30:11.9670004Z test_decomp.py::TestDecompCUDA::test_comprehensive_float_power_cuda_int32 PASSED [0.6256s] [ 19%] 2024-08-08T21:30:11.9671535Z test_decomp.py::TestDecompCUDA::test_comprehensive_floor_cuda_bfloat16 PASSED [0.0083s] [ 19%] 2024-08-08T21:30:11.9673026Z test_decomp.py::TestDecompCUDA::test_comprehensive_floor_cuda_int64 PASSED [0.0403s] [ 20%] 2024-08-08T21:30:11.9674501Z test_decomp.py::TestDecompCUDA::test_comprehensive_floor_cuda_int8 PASSED [0.0394s] [ 20%] 2024-08-08T21:30:11.9676055Z test_decomp.py::TestDecompCUDA::test_comprehensive_floor_cuda_uint8 PASSED [0.0395s] [ 20%] 2024-08-08T21:30:11.9677677Z test_decomp.py::TestDecompCUDA::test_comprehensive_floor_divide_cuda_float16 PASSED [0.1165s] [ 20%] 2024-08-08T21:30:11.9679226Z test_decomp.py::TestDecompCUDA::test_comprehensive_fmod_cuda_float32 PASSED [1.7885s] [ 20%] 2024-08-08T21:30:11.9680708Z test_decomp.py::TestDecompCUDA::test_comprehensive_frexp_cuda_float32 PASSED [0.2880s] [ 20%] 2024-08-08T21:30:11.9682221Z test_decomp.py::TestDecompCUDA::test_comprehensive_full_cuda_complex32 PASSED [0.0558s] [ 20%] 2024-08-08T21:30:11.9683712Z test_decomp.py::TestDecompCUDA::test_comprehensive_full_cuda_int16 PASSED [0.0222s] [ 20%] 2024-08-08T21:30:11.9685240Z test_decomp.py::TestDecompCUDA::test_comprehensive_full_like_cuda_complex128 PASSED [0.0105s] [ 20%] 2024-08-08T21:30:11.9686801Z test_decomp.py::TestDecompCUDA::test_comprehensive_full_like_cuda_float64 PASSED [0.0106s] [ 20%] 2024-08-08T21:30:11.9688341Z test_decomp.py::TestDecompCUDA::test_comprehensive_gather_cuda_bfloat16 PASSED [0.0303s] [ 20%] 2024-08-08T21:30:11.9689871Z test_decomp.py::TestDecompCUDA::test_comprehensive_gather_cuda_float64 PASSED [0.0848s] [ 20%] 2024-08-08T21:30:11.9691361Z test_decomp.py::TestDecompCUDA::test_comprehensive_gcd_cuda_uint8 PASSED [0.7059s] [ 21%] 2024-08-08T21:30:11.9692796Z test_decomp.py::TestDecompCUDA::test_comprehensive_gt_cuda_uint8 PASSED [0.1939s] [ 21%] 2024-08-08T21:30:11.9694267Z test_decomp.py::TestDecompCUDA::test_comprehensive_half_cuda_float64 PASSED [0.0888s] [ 21%] 2024-08-08T21:30:11.9695827Z test_decomp.py::TestDecompCUDA::test_comprehensive_half_cuda_int32 PASSED [0.0497s] [ 21%] 2024-08-08T21:30:11.9697401Z test_decomp.py::TestDecompCUDA::test_comprehensive_heaviside_cuda_float64 PASSED [3.2229s] [ 21%] 2024-08-08T21:30:11.9698959Z test_decomp.py::TestDecompCUDA::test_comprehensive_hsplit_cuda_bfloat16 PASSED [0.5884s] [ 21%] 2024-08-08T21:30:11.9700505Z test_decomp.py::TestDecompCUDA::test_comprehensive_hsplit_cuda_complex128 PASSED [2.6209s] [ 21%] 2024-08-08T21:30:11.9702030Z test_decomp.py::TestDecompCUDA::test_comprehensive_hsplit_cuda_int64 PASSED [0.0715s] [ 21%] 2024-08-08T21:30:11.9703526Z test_decomp.py::TestDecompCUDA::test_comprehensive_hstack_cuda_bfloat16 PASSED [0.0166s] [ 21%] 2024-08-08T21:30:11.9705076Z test_decomp.py::TestDecompCUDA::test_comprehensive_hstack_cuda_complex128 PASSED [0.1828s] [ 21%] 2024-08-08T21:30:11.9706616Z test_decomp.py::TestDecompCUDA::test_comprehensive_hstack_cuda_float64 PASSED [0.1067s] [ 21%] 2024-08-08T21:30:11.9708128Z test_decomp.py::TestDecompCUDA::test_comprehensive_hstack_cuda_uint8 PASSED [0.0312s] [ 21%] 2024-08-08T21:30:11.9709579Z test_decomp.py::TestDecompCUDA::test_comprehensive_i0_cuda_int16 PASSED [0.2533s] [ 21%] 2024-08-08T21:30:11.9711065Z test_decomp.py::TestDecompCUDA::test_comprehensive_igamma_cuda_float32 PASSED [0.2902s] [ 22%] 2024-08-08T21:30:11.9712587Z test_decomp.py::TestDecompCUDA::test_comprehensive_igammac_cuda_float64 PASSED [0.2967s] [ 22%] 2024-08-08T21:30:11.9714082Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_add_cuda_int8 PASSED [0.0884s] [ 22%] 2024-08-08T21:30:11.9715647Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_copy_cuda_complex32 PASSED [0.0781s] [ 22%] 2024-08-08T21:30:11.9717213Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_copy_cuda_int64 PASSED [0.0243s] [ 22%] 2024-08-08T21:30:11.9718820Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_copy_cuda_int8 PASSED [0.0246s] [ 22%] 2024-08-08T21:30:11.9720389Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_copy_cuda_uint8 PASSED [0.0240s] [ 22%] 2024-08-08T21:30:11.9721962Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_fill_cuda_complex64 PASSED [0.5736s] [ 22%] 2024-08-08T21:30:11.9723549Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_fill_cuda_float16 PASSED [0.0745s] [ 22%] 2024-08-08T21:30:11.9725089Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_put_cuda_int64 PASSED [0.0092s] [ 22%] 2024-08-08T21:30:11.9726652Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_amax_cuda_int64 PASSED [0.0123s] [ 22%] 2024-08-08T21:30:11.9728281Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_amin_cuda_int32 PASSED [0.0118s] [ 22%] 2024-08-08T21:30:11.9729911Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_amin_cuda_int8 PASSED [0.0118s] [ 23%] 2024-08-08T21:30:11.9731524Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_mean_cuda_float16 PASSED [0.3232s] [ 23%] 2024-08-08T21:30:11.9733159Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_mean_cuda_int32 PASSED [0.0133s] [ 23%] 2024-08-08T21:30:11.9734774Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_prod_cuda_int16 PASSED [0.0122s] [ 23%] 2024-08-08T21:30:11.9736389Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_reduce_prod_cuda_int32 PASSED [0.0117s] [ 23%] 2024-08-08T21:30:11.9737953Z test_decomp.py::TestDecompCUDA::test_comprehensive_index_select_cuda_int16 PASSED [0.0256s] [ 23%] 2024-08-08T21:30:11.9739571Z test_decomp.py::TestDecompCUDA::test_comprehensive_inner_cuda_complex128 PASSED [0.2379s] [ 23%] 2024-08-08T21:30:11.9741169Z test_decomp.py::TestDecompCUDA::test_comprehensive_inner_cuda_complex64 PASSED [0.2495s] [ 23%] 2024-08-08T21:30:11.9742696Z test_decomp.py::TestDecompCUDA::test_comprehensive_inner_cuda_float16 PASSED [0.0304s] [ 23%] 2024-08-08T21:30:11.9744177Z test_decomp.py::TestDecompCUDA::test_comprehensive_inner_cuda_float32 PASSED [0.1383s] [ 23%] 2024-08-08T21:30:11.9745644Z test_decomp.py::TestDecompCUDA::test_comprehensive_int_cuda_int16 PASSED [0.0320s] [ 23%] 2024-08-08T21:30:11.9747092Z test_decomp.py::TestDecompCUDA::test_comprehensive_int_cuda_int8 PASSED [0.0314s] [ 23%] 2024-08-08T21:30:11.9748583Z test_decomp.py::TestDecompCUDA::test_comprehensive_isclose_cuda_complex64 PASSED [5.3947s] [ 24%] 2024-08-08T21:30:11.9750128Z test_decomp.py::TestDecompCUDA::test_comprehensive_isclose_cuda_float32 PASSED [4.4708s] [ 24%] 2024-08-08T21:30:11.9751658Z test_decomp.py::TestDecompCUDA::test_comprehensive_isfinite_cuda_int64 PASSED [0.0114s] [ 24%] 2024-08-08T21:30:11.9753171Z test_decomp.py::TestDecompCUDA::test_comprehensive_isin_cuda_float64 PASSED [0.1138s] [ 24%] 2024-08-08T21:30:11.9754689Z test_decomp.py::TestDecompCUDA::test_comprehensive_isin_cuda_int16 PASSED [0.0775s] [ 24%] 2024-08-08T21:30:11.9756178Z test_decomp.py::TestDecompCUDA::test_comprehensive_isinf_cuda_float16 PASSED [0.0546s] [ 24%] 2024-08-08T21:30:11.9757668Z test_decomp.py::TestDecompCUDA::test_comprehensive_isinf_cuda_int64 PASSED [0.0725s] [ 24%] 2024-08-08T21:30:11.9759548Z test_decomp.py::TestDecompCUDA::test_comprehensive_isinf_cuda_uint8 PASSED [0.0726s] [ 24%] 2024-08-08T21:30:11.9761029Z test_decomp.py::TestDecompCUDA::test_comprehensive_isnan_cuda_float16 PASSED [0.0294s] [ 24%] 2024-08-08T21:30:11.9762516Z test_decomp.py::TestDecompCUDA::test_comprehensive_isnan_cuda_int64 PASSED [0.0396s] [ 24%] 2024-08-08T21:30:11.9764118Z test_decomp.py::TestDecompCUDA::test_comprehensive_isneginf_cuda_int16 PASSED [0.0729s] [ 24%] 2024-08-08T21:30:11.9765726Z test_decomp.py::TestDecompCUDA::test_comprehensive_isposinf_cuda_float32 PASSED [0.0982s] [ 24%] 2024-08-08T21:30:11.9767271Z test_decomp.py::TestDecompCUDA::test_comprehensive_isposinf_cuda_float64 PASSED [0.1029s] [ 25%] 2024-08-08T21:30:11.9768821Z test_decomp.py::TestDecompCUDA::test_comprehensive_isreal_cuda_complex128 PASSED [0.0175s] [ 25%] 2024-08-08T21:30:11.9770335Z test_decomp.py::TestDecompCUDA::test_comprehensive_isreal_cuda_int32 PASSED [0.0115s] [ 25%] 2024-08-08T21:30:11.9772116Z test_decomp.py::TestDecompCUDA::test_comprehensive_item_cuda_complex128 SKIPPED [0.0027s] (item in torch.complex128 not supported) [ 25%] 2024-08-08T21:30:11.9774038Z test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_2inputs_2outputs_cuda_float16 PASSED [0.5239s] [ 25%] 2024-08-08T21:30:11.9775845Z test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_4inputs_with_extra_args_cuda_float32 PASSED [0.5951s] [ 25%] 2024-08-08T21:30:11.9777569Z test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_binary_cuda_int32 PASSED [0.4757s] [ 25%] 2024-08-08T21:30:11.9779166Z test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_binary_cuda_uint8 PASSED [0.4707s] [ 25%] 2024-08-08T21:30:11.9780908Z test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_binary_return_by_ref_cuda_complex128 PASSED [0.5054s] [ 25%] 2024-08-08T21:30:11.9782617Z test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_unary_cuda_int16 PASSED [0.0660s] [ 25%] 2024-08-08T21:30:11.9784278Z test_decomp.py::TestDecompCUDA::test_comprehensive_jiterator_unary_cuda_uint8 PASSED [0.0626s] [ 25%] 2024-08-08T21:30:11.9785881Z test_decomp.py::TestDecompCUDA::test_comprehensive_kron_cuda_int32 PASSED [0.4231s] [ 25%] 2024-08-08T21:30:11.9787399Z test_decomp.py::TestDecompCUDA::test_comprehensive_kthvalue_cuda_bfloat16 PASSED [0.0604s] [ 25%] 2024-08-08T21:30:11.9788928Z test_decomp.py::TestDecompCUDA::test_comprehensive_kthvalue_cuda_int64 PASSED [0.0125s] [ 26%] 2024-08-08T21:30:11.9790387Z test_decomp.py::TestDecompCUDA::test_comprehensive_lcm_cuda_int32 PASSED [3.5044s] [ 26%] 2024-08-08T21:30:11.9791892Z test_decomp.py::TestDecompCUDA::test_comprehensive_ldexp_cuda_complex128 PASSED [4.1643s] [ 26%] 2024-08-08T21:30:11.9793376Z test_decomp.py::TestDecompCUDA::test_comprehensive_le_cuda_bool PASSED [0.1911s] [ 26%] 2024-08-08T21:30:11.9794811Z test_decomp.py::TestDecompCUDA::test_comprehensive_le_cuda_int16 PASSED [0.1923s] [ 26%] 2024-08-08T21:30:11.9796229Z test_decomp.py::TestDecompCUDA::test_comprehensive_le_cuda_int8 PASSED [0.1917s] [ 26%] 2024-08-08T21:30:11.9797668Z test_decomp.py::TestDecompCUDA::test_comprehensive_le_cuda_uint8 PASSED [0.1917s] [ 26%] 2024-08-08T21:30:11.9799161Z test_decomp.py::TestDecompCUDA::test_comprehensive_lgamma_cuda_bfloat16 PASSED [1.1173s] [ 26%] 2024-08-08T21:30:11.9800743Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_cholesky_cuda_complex64 PASSED [3.6950s] [ 26%] 2024-08-08T21:30:11.9802379Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_cond_cuda_complex128 PASSED [1.9037s] [ 26%] 2024-08-08T21:30:11.9803960Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_cross_cuda_int8 PASSED [0.3166s] [ 26%] 2024-08-08T21:30:11.9805541Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_det_cuda_complex128 PASSED [1.0813s] [ 26%] 2024-08-08T21:30:11.9807159Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_diagonal_cuda_complex64 PASSED [1.0714s] [ 27%] 2024-08-08T21:30:11.9808921Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_diagonal_cuda_int8 PASSED [0.1000s] [ 27%] 2024-08-08T21:30:11.9810566Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_eig_cuda_float32 PASSED [3.3233s] [ 27%] 2024-08-08T21:30:11.9812164Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_eigvals_cuda_float64 PASSED [0.4487s] [ 27%] 2024-08-08T21:30:11.9813874Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_householder_product_cuda_complex128 PASSED [28.5431s] [ 27%] 2024-08-08T21:30:11.9815631Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_ldl_factor_ex_cuda_complex64 PASSED [0.0180s] [ 27%] 2024-08-08T21:30:11.9817320Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_lu_factor_cuda_complex128 PASSED [45.4560s] [ 27%] 2024-08-08T21:30:11.9818984Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_lu_factor_ex_cuda_float64 PASSED [30.0398s] [ 27%] 2024-08-08T21:30:11.9820665Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_matrix_power_cuda_float64 PASSED [1.6554s] [ 27%] 2024-08-08T21:30:11.9822357Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_matrix_rank_cuda_complex128 PASSED [2.0338s] [ 27%] 2024-08-08T21:30:11.9824038Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_matrix_rank_cuda_float32 PASSED [1.9521s] [ 27%] 2024-08-08T21:30:11.9825766Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_matrix_rank_hermitian_cuda_complex128 PASSED [0.2568s] [ 27%] 2024-08-08T21:30:11.9827452Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_norm_cuda_float16 PASSED [1.8026s] [ 28%] 2024-08-08T21:30:11.9829222Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_norm_subgradients_at_zero_cuda_float64 PASSED [4.3015s] [ 28%] 2024-08-08T21:30:11.9830991Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_pinv_cuda_float32 PASSED [3.3512s] [ 28%] 2024-08-08T21:30:11.9833076Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_pinv_singular_cuda_complex64 SKIPPED [0.0014s] (test is slow; run with PYTORCH_TEST_WITH_SLOW to enable test) [ 28%] 2024-08-08T21:30:11.9835099Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_qr_cuda_complex128 PASSED [8.3375s] [ 28%] 2024-08-08T21:30:11.9836727Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_svdvals_cuda_complex64 PASSED [1.1055s] [ 28%] 2024-08-08T21:30:11.9838386Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_tensorsolve_cuda_float32 PASSED [0.5697s] [ 28%] 2024-08-08T21:30:11.9839989Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_vander_cuda_uint8 PASSED [3.2661s] [ 28%] 2024-08-08T21:30:11.9841634Z test_decomp.py::TestDecompCUDA::test_comprehensive_linalg_vector_norm_cuda_float64 PASSED [19.6681s] [ 28%] 2024-08-08T21:30:11.9843262Z test_decomp.py::TestDecompCUDA::test_comprehensive_linspace_cuda_complex64 PASSED [2.4507s] [ 28%] 2024-08-08T21:30:11.9844826Z test_decomp.py::TestDecompCUDA::test_comprehensive_linspace_cuda_float16 PASSED [0.4574s] [ 28%] 2024-08-08T21:30:11.9846458Z test_decomp.py::TestDecompCUDA::test_comprehensive_linspace_tensor_overload_cuda_float16 PASSED [1.6278s] [ 28%] 2024-08-08T21:30:11.9848186Z test_decomp.py::TestDecompCUDA::test_comprehensive_linspace_tensor_overload_cuda_int16 PASSED [6.3186s] [ 28%] 2024-08-08T21:30:11.9849789Z test_decomp.py::TestDecompCUDA::test_comprehensive_log10_cuda_float16 PASSED [0.0183s] [ 29%] 2024-08-08T21:30:11.9851297Z test_decomp.py::TestDecompCUDA::test_comprehensive_log1p_cuda_bfloat16 PASSED [0.0100s] [ 29%] 2024-08-08T21:30:11.9852860Z test_decomp.py::TestDecompCUDA::test_comprehensive_log1p_cuda_uint8 PASSED [0.0390s] [ 29%] 2024-08-08T21:30:11.9854423Z test_decomp.py::TestDecompCUDA::test_comprehensive_log2_cuda_int16 PASSED [0.0128s] [ 29%] 2024-08-08T21:30:11.9855883Z test_decomp.py::TestDecompCUDA::test_comprehensive_log2_cuda_int64 PASSED [0.0131s] [ 29%] 2024-08-08T21:30:11.9857315Z test_decomp.py::TestDecompCUDA::test_comprehensive_log_cuda_bool PASSED [0.0127s] [ 29%] 2024-08-08T21:30:11.9859161Z test_decomp.py::TestDecompCUDA::test_comprehensive_log_cuda_int16 PASSED [0.0125s] [ 29%] 2024-08-08T21:30:11.9861055Z test_decomp.py::TestDecompCUDA::test_comprehensive_log_cuda_int64 PASSED [0.0128s] [ 29%] 2024-08-08T21:30:11.9862714Z test_decomp.py::TestDecompCUDA::test_comprehensive_log_normal_cuda_float64 PASSED [0.0093s] [ 29%] 2024-08-08T21:30:11.9864280Z test_decomp.py::TestDecompCUDA::test_comprehensive_log_softmax_cuda_float32 PASSED [1.1429s] [ 29%] 2024-08-08T21:30:11.9865949Z test_decomp.py::TestDecompCUDA::test_comprehensive_log_softmax_with_dtype_cuda_complex64 PASSED [1.3249s] [ 29%] 2024-08-08T21:30:11.9867651Z test_decomp.py::TestDecompCUDA::test_comprehensive_log_softmax_with_dtype_cuda_int8 PASSED [0.8873s] [ 29%] 2024-08-08T21:30:11.9869240Z test_decomp.py::TestDecompCUDA::test_comprehensive_logdet_cuda_complex128 PASSED [2.2782s] [ 30%] 2024-08-08T21:30:11.9870805Z test_decomp.py::TestDecompCUDA::test_comprehensive_logical_not_cuda_float32 PASSED [0.0262s] [ 30%] 2024-08-08T21:30:11.9872388Z test_decomp.py::TestDecompCUDA::test_comprehensive_logical_or_cuda_complex64 PASSED [2.2388s] [ 30%] 2024-08-08T21:30:11.9874057Z test_decomp.py::TestDecompCUDA::test_comprehensive_logical_or_cuda_int16 PASSED [0.5598s] [ 30%] 2024-08-08T21:30:11.9875686Z test_decomp.py::TestDecompCUDA::test_comprehensive_logical_xor_cuda_bfloat16 PASSED [0.3786s] [ 30%] 2024-08-08T21:30:11.9877270Z test_decomp.py::TestDecompCUDA::test_comprehensive_logical_xor_cuda_float32 PASSED [0.7669s] [ 30%] 2024-08-08T21:30:11.9878805Z test_decomp.py::TestDecompCUDA::test_comprehensive_logit_cuda_float16 PASSED [0.4794s] [ 30%] 2024-08-08T21:30:11.9880336Z test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_cuda_float16 PASSED [3.5703s] [ 30%] 2024-08-08T21:30:11.9881842Z test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_cuda_int64 XFAIL [1.0675s] [ 30%] 2024-08-08T21:30:11.9883477Z test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_tensor_overload_cuda_float16 PASSED [13.4389s] [ 30%] 2024-08-08T21:30:11.9885220Z test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_tensor_overload_cuda_float64 PASSED [46.6481s] [ 30%] 2024-08-08T21:30:11.9886941Z test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_tensor_overload_cuda_int16 XFAIL [4.6865s] [ 30%] 2024-08-08T21:30:11.9888642Z test_decomp.py::TestDecompCUDA::test_comprehensive_logspace_tensor_overload_cuda_int32 XFAIL [4.6745s] [ 31%] 2024-08-08T21:30:11.9890232Z test_decomp.py::TestDecompCUDA::test_comprehensive_long_cuda_bfloat16 PASSED [0.0301s] [ 31%] 2024-08-08T21:30:11.9891749Z test_decomp.py::TestDecompCUDA::test_comprehensive_long_cuda_complex128 PASSED [0.0594s] [ 31%] 2024-08-08T21:30:11.9893225Z test_decomp.py::TestDecompCUDA::test_comprehensive_long_cuda_int32 PASSED [0.0309s] [ 31%] 2024-08-08T21:30:11.9894697Z test_decomp.py::TestDecompCUDA::test_comprehensive_lu_cuda_float32 PASSED [32.9579s] [ 31%] 2024-08-08T21:30:11.9896178Z test_decomp.py::TestDecompCUDA::test_comprehensive_mH_cuda_bfloat16 PASSED [0.0263s] [ 31%] 2024-08-08T21:30:11.9897758Z test_decomp.py::TestDecompCUDA::test_comprehensive_mH_cuda_complex128 PASSED [1.8127s] [ 31%] 2024-08-08T21:30:11.9899279Z test_decomp.py::TestDecompCUDA::test_comprehensive_mH_cuda_int8 PASSED [0.3437s] [ 31%] 2024-08-08T21:30:11.9900707Z test_decomp.py::TestDecompCUDA::test_comprehensive_mT_cuda_bool PASSED [0.3403s] [ 31%] 2024-08-08T21:30:11.9902172Z test_decomp.py::TestDecompCUDA::test_comprehensive_mT_cuda_complex32 PASSED [0.7791s] [ 31%] 2024-08-08T21:30:11.9903603Z test_decomp.py::TestDecompCUDA::test_comprehensive_mT_cuda_int8 PASSED [0.3441s] [ 31%] 2024-08-08T21:30:11.9905106Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_amax_cuda_float16 PASSED [1.6190s] [ 31%] 2024-08-08T21:30:11.9906685Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_amax_cuda_float64 PASSED [5.0513s] [ 32%] 2024-08-08T21:30:11.9908276Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_amin_cuda_bfloat16 PASSED [1.6227s] [ 32%] 2024-08-08T21:30:11.9909841Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_amin_cuda_float16 PASSED [1.6186s] [ 32%] 2024-08-08T21:30:11.9911435Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_argmax_cuda_float16 PASSED [0.3070s] [ 32%] 2024-08-08T21:30:11.9913025Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_argmax_cuda_int32 PASSED [0.5429s] [ 32%] 2024-08-08T21:30:11.9914626Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_argmin_cuda_bfloat16 PASSED [0.3057s] [ 32%] 2024-08-08T21:30:11.9916210Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_cumprod_cuda_int16 PASSED [2.9465s] [ 32%] 2024-08-08T21:30:11.9917850Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_cumsum_cuda_int32 PASSED [2.9556s] [ 32%] 2024-08-08T21:30:11.9919481Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_cumsum_cuda_int8 PASSED [2.9476s] [ 32%] 2024-08-08T21:30:11.9921027Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_fill_cuda_int16 PASSED [0.1404s] [ 32%] 2024-08-08T21:30:11.9922576Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_fill_cuda_int8 PASSED [0.1384s] [ 32%] 2024-08-08T21:30:11.9924180Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_log_softmax_cuda_float32 PASSED [5.8859s] [ 32%] 2024-08-08T21:30:11.9925800Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_mean_cuda_bfloat16 PASSED [1.2076s] [ 32%] 2024-08-08T21:30:11.9927392Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_mean_cuda_complex128 PASSED [6.0270s] [ 33%] 2024-08-08T21:30:11.9928988Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_mean_cuda_float16 PASSED [1.2070s] [ 33%] 2024-08-08T21:30:11.9930567Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_mean_cuda_float32 PASSED [4.1541s] [ 33%] 2024-08-08T21:30:11.9932216Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_normalize_cuda_complex128 PASSED [24.2061s] [ 33%] 2024-08-08T21:30:11.9933876Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_normalize_cuda_float64 PASSED [16.6490s] [ 33%] 2024-08-08T21:30:11.9935473Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_prod_cuda_bool PASSED [2.4218s] [ 33%] 2024-08-08T21:30:11.9937020Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_prod_cuda_int32 PASSED [1.3638s] [ 33%] 2024-08-08T21:30:11.9938554Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_prod_cuda_uint8 PASSED [1.3615s] [ 33%] 2024-08-08T21:30:11.9940144Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_scatter_cuda_float32 PASSED [0.1450s] [ 33%] 2024-08-08T21:30:11.9941834Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_scatter_cuda_int64 PASSED [0.0088s] [ 33%] 2024-08-08T21:30:11.9943489Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_select_cuda_bfloat16 PASSED [0.0335s] [ 33%] 2024-08-08T21:30:11.9945080Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_select_cuda_float16 PASSED [0.0339s] [ 33%] 2024-08-08T21:30:11.9946666Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_select_cuda_int8 PASSED [0.0115s] [ 34%] 2024-08-08T21:30:11.9948239Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_select_cuda_uint8 PASSED [0.0115s] [ 34%] 2024-08-08T21:30:11.9949845Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_softmax_cuda_float16 PASSED [0.4437s] [ 34%] 2024-08-08T21:30:11.9951445Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_softmax_cuda_float32 PASSED [5.8413s] [ 34%] 2024-08-08T21:30:11.9953039Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_sum_cuda_float16 PASSED [0.7486s] [ 34%] 2024-08-08T21:30:11.9954698Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_var_cuda_complex64 PASSED [31.5518s] [ 34%] 2024-08-08T21:30:11.9956253Z test_decomp.py::TestDecompCUDA::test_comprehensive_masked_var_cuda_int64 PASSED [8.0289s] [ 34%] 2024-08-08T21:30:11.9957780Z test_decomp.py::TestDecompCUDA::test_comprehensive_matmul_cuda_float64 PASSED [5.7220s] [ 34%] 2024-08-08T21:30:11.9959735Z test_decomp.py::TestDecompCUDA::test_comprehensive_max_binary_cuda_int16 PASSED [0.1884s] [ 34%] 2024-08-08T21:30:11.9961274Z test_decomp.py::TestDecompCUDA::test_comprehensive_max_binary_cuda_int32 PASSED [0.1884s] [ 34%] 2024-08-08T21:30:11.9962945Z test_decomp.py::TestDecompCUDA::test_comprehensive_max_reduction_no_dim_cuda_bool PASSED [0.0063s] [ 34%] 2024-08-08T21:30:11.9964697Z test_decomp.py::TestDecompCUDA::test_comprehensive_max_reduction_no_dim_cuda_float32 PASSED [0.1612s] [ 34%] 2024-08-08T21:30:11.9966368Z test_decomp.py::TestDecompCUDA::test_comprehensive_max_reduction_with_dim_cuda_int32 PASSED [0.0077s] [ 35%] 2024-08-08T21:30:11.9967966Z test_decomp.py::TestDecompCUDA::test_comprehensive_maximum_cuda_float16 PASSED [0.8952s] [ 35%] 2024-08-08T21:30:11.9969470Z test_decomp.py::TestDecompCUDA::test_comprehensive_maximum_cuda_float64 PASSED [8.5223s] [ 35%] 2024-08-08T21:30:11.9970987Z test_decomp.py::TestDecompCUDA::test_comprehensive_maximum_cuda_int16 PASSED [0.1904s] [ 35%] 2024-08-08T21:30:11.9972498Z test_decomp.py::TestDecompCUDA::test_comprehensive_mean_cuda_float32 PASSED [0.4380s] [ 35%] 2024-08-08T21:30:11.9973972Z test_decomp.py::TestDecompCUDA::test_comprehensive_median_cuda_int32 PASSED [0.0151s] [ 35%] 2024-08-08T21:30:11.9975922Z test_decomp.py::TestDecompCUDA::test_comprehensive_meshgrid_list_of_tensors_cuda_bfloat16 SKIPPED [0.0027s] (meshgrid in torch.bfloat16 not supported) [ 35%] 2024-08-08T21:30:11.9978254Z test_decomp.py::TestDecompCUDA::test_comprehensive_meshgrid_list_of_tensors_cuda_complex64 SKIPPED [0.0026s] (meshgrid in torch.complex64 not supported) [ 35%] 2024-08-08T21:30:11.9980528Z test_decomp.py::TestDecompCUDA::test_comprehensive_meshgrid_list_of_tensors_cuda_uint8 SKIPPED [0.0030s] (meshgrid in torch.uint8 not supported) [ 35%] 2024-08-08T21:30:11.9982810Z test_decomp.py::TestDecompCUDA::test_comprehensive_meshgrid_variadic_tensors_cuda_bfloat16 SKIPPED [0.0026s] (meshgrid in torch.bfloat16 not supported) [ 35%] 2024-08-08T21:30:11.9984693Z test_decomp.py::TestDecompCUDA::test_comprehensive_min_binary_cuda_bool PASSED [0.1889s] [ 35%] 2024-08-08T21:30:11.9986385Z test_decomp.py::TestDecompCUDA::test_comprehensive_min_reduction_no_dim_cuda_float32 PASSED [0.1607s] [ 35%] 2024-08-08T21:30:11.9988153Z test_decomp.py::TestDecompCUDA::test_comprehensive_min_reduction_with_dim_cuda_float16 PASSED [0.0230s] [ 35%] 2024-08-08T21:30:11.9989723Z test_decomp.py::TestDecompCUDA::test_comprehensive_minimum_cuda_bool PASSED [0.1868s] [ 36%] 2024-08-08T21:30:11.9991233Z test_decomp.py::TestDecompCUDA::test_comprehensive_minimum_cuda_float32 PASSED [8.4540s] [ 36%] 2024-08-08T21:30:11.9992746Z test_decomp.py::TestDecompCUDA::test_comprehensive_minimum_cuda_int32 PASSED [0.1869s] [ 36%] 2024-08-08T21:30:11.9994247Z test_decomp.py::TestDecompCUDA::test_comprehensive_mm_cuda_complex64 PASSED [0.2774s] [ 36%] 2024-08-08T21:30:11.9995728Z test_decomp.py::TestDecompCUDA::test_comprehensive_mode_cuda_bfloat16 PASSED [0.0542s] [ 36%] 2024-08-08T21:30:11.9997242Z test_decomp.py::TestDecompCUDA::test_comprehensive_movedim_cuda_float16 PASSED [0.0103s] [ 36%] 2024-08-08T21:30:11.9998774Z test_decomp.py::TestDecompCUDA::test_comprehensive_movedim_cuda_int32 PASSED [0.0225s] [ 36%] 2024-08-08T21:30:12.0000261Z test_decomp.py::TestDecompCUDA::test_comprehensive_msort_cuda_int64 PASSED [0.0068s] [ 36%] 2024-08-08T21:30:12.0001709Z test_decomp.py::TestDecompCUDA::test_comprehensive_mul_cuda_int16 PASSED [0.1847s] [ 36%] 2024-08-08T21:30:12.0003178Z test_decomp.py::TestDecompCUDA::test_comprehensive_mv_cuda_complex64 PASSED [0.2774s] [ 36%] 2024-08-08T21:30:12.0004780Z test_decomp.py::TestDecompCUDA::test_comprehensive_mvlgamma_mvlgamma_p_1_cuda_float16 PASSED [0.7269s] [ 36%] 2024-08-08T21:30:12.0006442Z test_decomp.py::TestDecompCUDA::test_comprehensive_mvlgamma_mvlgamma_p_1_cuda_int8 PASSED [0.6816s] [ 36%] 2024-08-08T21:30:12.0008157Z test_decomp.py::TestDecompCUDA::test_comprehensive_mvlgamma_mvlgamma_p_5_cuda_int8 PASSED [0.6792s] [ 37%] 2024-08-08T21:30:12.0009821Z test_decomp.py::TestDecompCUDA::test_comprehensive_nanmean_cuda_complex32 PASSED [12.0694s] [ 37%] 2024-08-08T21:30:12.0011389Z test_decomp.py::TestDecompCUDA::test_comprehensive_nanmean_cuda_complex64 PASSED [8.3834s] [ 37%] 2024-08-08T21:30:12.0012917Z test_decomp.py::TestDecompCUDA::test_comprehensive_nansum_cuda_bfloat16 PASSED [0.6913s] [ 37%] 2024-08-08T21:30:12.0014425Z test_decomp.py::TestDecompCUDA::test_comprehensive_nansum_cuda_int32 PASSED [0.4709s] [ 37%] 2024-08-08T21:30:12.0015959Z test_decomp.py::TestDecompCUDA::test_comprehensive_narrow_copy_cuda_float16 PASSED [0.0150s] [ 37%] 2024-08-08T21:30:12.0017520Z test_decomp.py::TestDecompCUDA::test_comprehensive_narrow_copy_cuda_int64 PASSED [0.1120s] [ 37%] 2024-08-08T21:30:12.0019050Z test_decomp.py::TestDecompCUDA::test_comprehensive_narrow_cuda_complex64 PASSED [3.7865s] [ 37%] 2024-08-08T21:30:12.0020568Z test_decomp.py::TestDecompCUDA::test_comprehensive_narrow_cuda_int32 PASSED [0.1602s] [ 37%] 2024-08-08T21:30:12.0022026Z test_decomp.py::TestDecompCUDA::test_comprehensive_ne_cuda_bool PASSED [0.1845s] [ 37%] 2024-08-08T21:30:12.0023444Z test_decomp.py::TestDecompCUDA::test_comprehensive_ne_cuda_int32 PASSED [0.1866s] [ 37%] 2024-08-08T21:30:12.0024917Z test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_complex32 PASSED [0.1683s] [ 37%] 2024-08-08T21:30:12.0026407Z test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_float16 PASSED [0.0082s] [ 38%] 2024-08-08T21:30:12.0027868Z test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_int32 PASSED [0.0390s] [ 38%] 2024-08-08T21:30:12.0029301Z test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_int8 PASSED [0.0385s] [ 38%] 2024-08-08T21:30:12.0030809Z test_decomp.py::TestDecompCUDA::test_comprehensive_neg_cuda_uint8 PASSED [0.0385s] [ 38%] 2024-08-08T21:30:12.0032732Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_empty_strided_cuda_bool SKIPPED [0.0003s] (Expected: new_empty_strided is not comparable) [ 38%] 2024-08-08T21:30:12.0034976Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_empty_strided_cuda_float32 SKIPPED [0.0002s] (Expected: new_empty_strided is not comparable) [ 38%] 2024-08-08T21:30:12.0037181Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_empty_strided_cuda_int8 SKIPPED [0.0002s] (Expected: new_empty_strided is not comparable) [ 38%] 2024-08-08T21:30:12.0039005Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_full_cuda_bool PASSED [0.0486s] [ 38%] 2024-08-08T21:30:12.0040529Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_full_cuda_float32 PASSED [0.0812s] [ 38%] 2024-08-08T21:30:12.0042075Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_full_cuda_float64 PASSED [0.0803s] [ 38%] 2024-08-08T21:30:12.0043603Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_ones_cuda_bfloat16 PASSED [0.0262s] [ 38%] 2024-08-08T21:30:12.0045402Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_ones_cuda_complex32 PASSED [0.1177s] [ 38%] 2024-08-08T21:30:12.0047155Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_ones_cuda_float32 PASSED [0.0814s] [ 39%] 2024-08-08T21:30:12.0048668Z test_decomp.py::TestDecompCUDA::test_comprehensive_new_zeros_cuda_int16 PASSED [0.0476s] [ 39%] 2024-08-08T21:30:12.0050373Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_adaptive_avg_pool1d_cuda_float32 PASSED [2.2423s] [ 39%] 2024-08-08T21:30:12.0052305Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_adaptive_avg_pool2d_cuda_float64 PASSED [10.9106s] [ 39%] 2024-08-08T21:30:12.0054247Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_alpha_dropout_cuda_float32 PASSED [0.2407s] [ 39%] 2024-08-08T21:30:12.0055993Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_avg_pool1d_cuda_float32 PASSED [0.3410s] [ 39%] 2024-08-08T21:30:12.0057819Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_batch_norm_without_cudnn_cuda_float32 PASSED [4.4310s] [ 39%] 2024-08-08T21:30:12.0060252Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_binary_cross_entropy_with_logits_cuda_float16 PASSED [0.7910s] [ 39%] 2024-08-08T21:30:12.0062165Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_channel_shuffle_cuda_complex64 PASSED [2.2568s] [ 39%] 2024-08-08T21:30:12.0063963Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_channel_shuffle_cuda_float16 PASSED [0.0246s] [ 39%] 2024-08-08T21:30:12.0065756Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_channel_shuffle_cuda_int8 PASSED [0.4812s] [ 39%] 2024-08-08T21:30:12.0067507Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_conv1d_cuda_complex128 PASSED [6.2455s] [ 39%] 2024-08-08T21:30:12.0069223Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_conv2d_cuda_complex64 PASSED [55.9218s] [ 39%] 2024-08-08T21:30:12.0070902Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_conv3d_cuda_float64 PASSED [2.1259s] [ 40%] 2024-08-08T21:30:12.0072675Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_conv_transpose2d_cuda_complex32 PASSED [4.6366s] [ 40%] 2024-08-08T21:30:12.0074517Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_cosine_embedding_loss_cuda_bool PASSED [0.5164s] [ 40%] 2024-08-08T21:30:12.0076461Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_cosine_similarity_cuda_bfloat16 PASSED [1.0077s] [ 40%] 2024-08-08T21:30:12.0078342Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_cross_entropy_cuda_float32 PASSED [5.3607s] [ 40%] 2024-08-08T21:30:12.0080092Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_dropout2d_cuda_float32 PASSED [0.8848s] [ 40%] 2024-08-08T21:30:12.0081799Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_dropout_cuda_float64 PASSED [0.2474s] [ 40%] 2024-08-08T21:30:12.0083515Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_embedding_cuda_bfloat16 PASSED [0.0743s] [ 40%] 2024-08-08T21:30:12.0085301Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_fractional_max_pool2d_cuda_float16 PASSED [0.0729s] [ 40%] 2024-08-08T21:30:12.0087161Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_fractional_max_pool2d_cuda_float32 PASSED [0.0721s] [ 40%] 2024-08-08T21:30:12.0088934Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_gelu_cuda_float16 PASSED [0.2195s] [ 40%] 2024-08-08T21:30:12.0090567Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_glu_cuda_float16 PASSED [0.7038s] [ 40%] 2024-08-08T21:30:12.0092284Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_grid_sample_cuda_float16 PASSED [71.4440s] [ 41%] 2024-08-08T21:30:12.0094475Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_hardshrink_cuda_float64 SKIPPED [0.0028s] (nn.functional.hardshrink in torch.float64 not supported) [ 41%] 2024-08-08T21:30:12.0096578Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_hardswish_cuda_float32 PASSED [2.6072s] [ 41%] 2024-08-08T21:30:12.0098344Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_hardtanh_cuda_int32 PASSED [0.2729s] [ 41%] 2024-08-08T21:30:12.0100266Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_huber_loss_cuda_float16 PASSED [0.6334s] [ 41%] 2024-08-08T21:30:12.0102018Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_instance_norm_cuda_float32 PASSED [7.0048s] [ 41%] 2024-08-08T21:30:12.0103811Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_area_cuda_float16 PASSED [0.8751s] [ 41%] 2024-08-08T21:30:12.0105642Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_bilinear_cuda_bfloat16 PASSED [3.2439s] [ 41%] 2024-08-08T21:30:12.0107493Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_linear_cuda_bfloat16 PASSED [1.4620s] [ 41%] 2024-08-08T21:30:12.0109415Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_linear_cuda_float32 PASSED [4.2440s] [ 41%] 2024-08-08T21:30:12.0111309Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_nearest-exact_cuda_float16 PASSED [0.5426s] [ 41%] 2024-08-08T21:30:12.0113202Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_interpolate_trilinear_cuda_float16 PASSED [5.2944s] [ 41%] 2024-08-08T21:30:12.0114961Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_kl_div_cuda_float32 PASSED [2.8006s] [ 42%] 2024-08-08T21:30:12.0116648Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_l1_loss_cuda_float64 PASSED [0.5035s] [ 42%] 2024-08-08T21:30:12.0118362Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_layer_norm_cuda_float64 PASSED [1.5596s] [ 42%] 2024-08-08T21:30:12.0120048Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_linear_cuda_float64 PASSED [2.8683s] [ 42%] 2024-08-08T21:30:12.0121883Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_local_response_norm_cuda_bfloat16 PASSED [0.3658s] [ 42%] 2024-08-08T21:30:12.0123783Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_local_response_norm_cuda_float32 PASSED [2.6890s] [ 42%] 2024-08-08T21:30:12.0125619Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_margin_ranking_loss_cuda_float16 PASSED [0.4514s] [ 42%] 2024-08-08T21:30:12.0127387Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_pool1d_cuda_float16 PASSED [7.5980s] [ 42%] 2024-08-08T21:30:12.0129105Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_pool3d_cuda_float64 PASSED [0.7218s] [ 42%] 2024-08-08T21:30:12.0130886Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_unpool1d_grad_cuda_float64 PASSED [0.6852s] [ 42%] 2024-08-08T21:30:12.0132710Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_unpool2d_grad_cuda_float16 PASSED [0.2618s] [ 42%] 2024-08-08T21:30:12.0134474Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_max_unpool3d_cuda_float32 PASSED [8.1262s] [ 42%] 2024-08-08T21:30:12.0136220Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_mse_loss_cuda_bfloat16 PASSED [0.1105s] [ 42%] 2024-08-08T21:30:12.0137929Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_mse_loss_cuda_float32 PASSED [1.0307s] [ 43%] 2024-08-08T21:30:12.0139726Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_multilabel_margin_loss_cuda_float16 PASSED [1.2930s] [ 43%] 2024-08-08T21:30:12.0143889Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_nll_loss_cuda_float32 SKIPPED [0.0005s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/117732 for platform(s) rocm, win, windows, linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 43%] 2024-08-08T21:30:12.0148059Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_circular_cuda_float16 SKIPPED [0.0002s] (Expected: new_empty_strided is not comparable) [ 43%] 2024-08-08T21:30:12.0150078Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_constant_cuda_bool PASSED [1.0657s] [ 43%] 2024-08-08T21:30:12.0151835Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_constant_cuda_float64 PASSED [2.8786s] [ 43%] 2024-08-08T21:30:12.0153562Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_constant_cuda_int16 PASSED [1.0710s] [ 43%] 2024-08-08T21:30:12.0155376Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_reflect_cuda_complex64 PASSED [61.4729s] [ 43%] 2024-08-08T21:30:12.0157155Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_replicate_cuda_int32 PASSED [1.3003s] [ 43%] 2024-08-08T21:30:12.0159412Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pad_replicate_negative_cuda_uint8 PASSED [1.2747s] [ 43%] 2024-08-08T21:30:12.0161268Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pairwise_distance_cuda_int64 PASSED [0.2662s] [ 43%] 2024-08-08T21:30:12.0163081Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pixel_shuffle_cuda_complex128 PASSED [0.3163s] [ 43%] 2024-08-08T21:30:12.0164868Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pixel_shuffle_cuda_int32 PASSED [0.0777s] [ 44%] 2024-08-08T21:30:12.0166655Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pixel_unshuffle_cuda_bfloat16 PASSED [0.0352s] [ 44%] 2024-08-08T21:30:12.0168565Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_pixel_unshuffle_cuda_complex128 PASSED [0.2927s] [ 44%] 2024-08-08T21:30:12.0170468Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_poisson_nll_loss_cuda_float32 PASSED [37.3924s] [ 44%] 2024-08-08T21:30:12.0172266Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_poisson_nll_loss_cuda_uint8 PASSED [13.4854s] [ 44%] 2024-08-08T21:30:12.0173994Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_prelu_cuda_float64 PASSED [8.4763s] [ 44%] 2024-08-08T21:30:12.0176044Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_relu6_cuda_bfloat16 SKIPPED [0.0028s] (nn.functional.relu6 in torch.bfloat16 not supported) [ 44%] 2024-08-08T21:30:12.0178418Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_relu6_cuda_int16 SKIPPED [0.0027s] (nn.functional.relu6 in torch.int16 not supported) [ 44%] 2024-08-08T21:30:12.0180635Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_scaled_dot_product_attention_cuda_bfloat16 SKIPPED [0.0002s] (Skipped) [ 44%] 2024-08-08T21:30:12.0182733Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_scaled_dot_product_attention_cuda_float32 SKIPPED [0.0003s] (Skipped) [ 44%] 2024-08-08T21:30:12.0184609Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softmin_cuda_bfloat16 PASSED [0.1211s] [ 44%] 2024-08-08T21:30:12.0186395Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softmin_with_dtype_cuda_float16 PASSED [0.1233s] [ 44%] 2024-08-08T21:30:12.0188163Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softplus_cuda_float16 PASSED [0.0590s] [ 45%] 2024-08-08T21:30:12.0189930Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softsign_cuda_bool PASSED [0.0276s] [ 45%] 2024-08-08T21:30:12.0191692Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softsign_cuda_float16 PASSED [0.0475s] [ 45%] 2024-08-08T21:30:12.0193397Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softsign_cuda_float64 PASSED [0.1888s] [ 45%] 2024-08-08T21:30:12.0195094Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_softsign_cuda_int32 PASSED [0.0276s] [ 45%] 2024-08-08T21:30:12.0196790Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_tanhshrink_cuda_uint8 PASSED [0.0279s] [ 45%] 2024-08-08T21:30:12.0198476Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_threshold_cuda_int32 PASSED [0.1519s] [ 45%] 2024-08-08T21:30:12.0200179Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_threshold_cuda_int64 PASSED [0.1432s] [ 45%] 2024-08-08T21:30:12.0202051Z test_decomp.py::TestDecompCUDA::test_comprehensive_nn_functional_triplet_margin_with_distance_loss_cuda_float16 PASSED [0.9669s] [ 45%] 2024-08-08T21:30:12.0203806Z test_decomp.py::TestDecompCUDA::test_comprehensive_nonzero_cuda_float64 PASSED [0.3628s] [ 45%] 2024-08-08T21:30:12.0205298Z test_decomp.py::TestDecompCUDA::test_comprehensive_nonzero_cuda_uint8 PASSED [0.3288s] [ 45%] 2024-08-08T21:30:12.0206989Z test_decomp.py::TestDecompCUDA::test_comprehensive_nonzero_static_cuda_int64 SKIPPED [0.0013s] (Only runs on cpu) [ 45%] 2024-08-08T21:30:12.0208942Z test_decomp.py::TestDecompCUDA::test_comprehensive_norm_cuda_complex64 SKIPPED [0.0026s] (norm in torch.complex64 not supported) [ 46%] 2024-08-08T21:30:12.0211017Z test_decomp.py::TestDecompCUDA::test_comprehensive_norm_fro_cuda_complex64 SKIPPED [0.0026s] (norm in torch.complex64 not supported) [ 46%] 2024-08-08T21:30:12.0212922Z test_decomp.py::TestDecompCUDA::test_comprehensive_normal_cuda_float16 SKIPPED [0.0002s] (Skipped!) [ 46%] 2024-08-08T21:30:12.0214629Z test_decomp.py::TestDecompCUDA::test_comprehensive_normal_in_place_cuda_float64 PASSED [0.0079s] [ 46%] 2024-08-08T21:30:12.0216188Z test_decomp.py::TestDecompCUDA::test_comprehensive_ones_cuda_float16 PASSED [0.0109s] [ 46%] 2024-08-08T21:30:12.0217650Z test_decomp.py::TestDecompCUDA::test_comprehensive_ones_cuda_float32 PASSED [0.0232s] [ 46%] 2024-08-08T21:30:12.0219121Z test_decomp.py::TestDecompCUDA::test_comprehensive_ones_cuda_int8 PASSED [0.0147s] [ 46%] 2024-08-08T21:30:12.0220629Z test_decomp.py::TestDecompCUDA::test_comprehensive_ones_like_cuda_float16 PASSED [0.0160s] [ 46%] 2024-08-08T21:30:12.0222138Z test_decomp.py::TestDecompCUDA::test_comprehensive_outer_cuda_bool PASSED [0.0162s] [ 46%] 2024-08-08T21:30:12.0223679Z test_decomp.py::TestDecompCUDA::test_comprehensive_pca_lowrank_cuda_complex64 PASSED [71.3424s] [ 46%] 2024-08-08T21:30:12.0225268Z test_decomp.py::TestDecompCUDA::test_comprehensive_permute_cuda_complex32 PASSED [0.0740s] [ 46%] 2024-08-08T21:30:12.0226837Z test_decomp.py::TestDecompCUDA::test_comprehensive_pinverse_cuda_complex64 PASSED [2.4444s] [ 46%] 2024-08-08T21:30:12.0228499Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_0_cuda_float32 PASSED [0.1152s] [ 46%] 2024-08-08T21:30:12.0230198Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_0_cuda_float64 PASSED [0.1129s] [ 47%] 2024-08-08T21:30:12.0231902Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_0_cuda_int16 PASSED [0.9725s] [ 47%] 2024-08-08T21:30:12.0233679Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_1_cuda_bfloat16 PASSED [1.3228s] [ 47%] 2024-08-08T21:30:12.0235449Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_2_cuda_float64 PASSED [0.1105s] [ 47%] 2024-08-08T21:30:12.0237151Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_2_cuda_int8 PASSED [0.0114s] [ 47%] 2024-08-08T21:30:12.0238835Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_3_cuda_int64 PASSED [0.0110s] [ 47%] 2024-08-08T21:30:12.0240525Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_3_cuda_int8 PASSED [0.0109s] [ 47%] 2024-08-08T21:30:12.0242186Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_3_cuda_uint8 PASSED [0.0113s] [ 47%] 2024-08-08T21:30:12.0243871Z test_decomp.py::TestDecompCUDA::test_comprehensive_polygamma_polygamma_n_4_cuda_bool PASSED [0.0110s] [ 47%] 2024-08-08T21:30:12.0245504Z test_decomp.py::TestDecompCUDA::test_comprehensive_positive_cuda_complex64 PASSED [0.0056s] [ 47%] 2024-08-08T21:30:12.0247074Z test_decomp.py::TestDecompCUDA::test_comprehensive_positive_cuda_float16 PASSED [0.0062s] [ 47%] 2024-08-08T21:30:12.0248569Z test_decomp.py::TestDecompCUDA::test_comprehensive_pow_cuda_float64 PASSED [4.3573s] [ 47%] 2024-08-08T21:30:12.0250031Z test_decomp.py::TestDecompCUDA::test_comprehensive_pow_cuda_int8 PASSED [0.1949s] [ 48%] 2024-08-08T21:30:12.0251496Z test_decomp.py::TestDecompCUDA::test_comprehensive_prod_cuda_float16 PASSED [3.8775s] [ 48%] 2024-08-08T21:30:12.0252971Z test_decomp.py::TestDecompCUDA::test_comprehensive_prod_cuda_int64 PASSED [0.3994s] [ 48%] 2024-08-08T21:30:12.0254497Z test_decomp.py::TestDecompCUDA::test_comprehensive_put_cuda_complex64 PASSED [0.5301s] [ 48%] 2024-08-08T21:30:12.0256003Z test_decomp.py::TestDecompCUDA::test_comprehensive_rad2deg_cuda_int64 PASSED [0.1464s] [ 48%] 2024-08-08T21:30:12.0257566Z test_decomp.py::TestDecompCUDA::test_comprehensive_randint_cuda_int16 PASSED [0.0247s] [ 48%] 2024-08-08T21:30:12.0259550Z test_decomp.py::TestDecompCUDA::test_comprehensive_randint_cuda_int32 PASSED [0.0245s] [ 48%] 2024-08-08T21:30:12.0261053Z test_decomp.py::TestDecompCUDA::test_comprehensive_randint_cuda_uint8 PASSED [0.0241s] [ 48%] 2024-08-08T21:30:12.0262609Z test_decomp.py::TestDecompCUDA::test_comprehensive_randint_like_cuda_float16 PASSED [0.0252s] [ 48%] 2024-08-08T21:30:12.0264202Z test_decomp.py::TestDecompCUDA::test_comprehensive_randint_like_cuda_float32 PASSED [0.0246s] [ 48%] 2024-08-08T21:30:12.0265751Z test_decomp.py::TestDecompCUDA::test_comprehensive_randint_like_cuda_uint8 PASSED [0.0246s] [ 48%] 2024-08-08T21:30:12.0267305Z test_decomp.py::TestDecompCUDA::test_comprehensive_randn_cuda_complex64 PASSED [0.0075s] [ 48%] 2024-08-08T21:30:12.0268869Z test_decomp.py::TestDecompCUDA::test_comprehensive_randn_like_cuda_bfloat16 PASSED [0.0148s] [ 49%] 2024-08-08T21:30:12.0270371Z test_decomp.py::TestDecompCUDA::test_comprehensive_ravel_cuda_bool PASSED [0.0595s] [ 49%] 2024-08-08T21:30:12.0271861Z test_decomp.py::TestDecompCUDA::test_comprehensive_ravel_cuda_float32 PASSED [0.1164s] [ 49%] 2024-08-08T21:30:12.0273342Z test_decomp.py::TestDecompCUDA::test_comprehensive_ravel_cuda_int64 PASSED [0.0598s] [ 49%] 2024-08-08T21:30:12.0274842Z test_decomp.py::TestDecompCUDA::test_comprehensive_real_cuda_complex128 PASSED [0.1330s] [ 49%] 2024-08-08T21:30:12.0276369Z test_decomp.py::TestDecompCUDA::test_comprehensive_reciprocal_cuda_float64 PASSED [0.0639s] [ 49%] 2024-08-08T21:30:12.0278041Z test_decomp.py::TestDecompCUDA::test_comprehensive_reciprocal_cuda_uint8 PASSED [0.0132s] [ 49%] 2024-08-08T21:30:12.0279680Z test_decomp.py::TestDecompCUDA::test_comprehensive_remainder_cuda_float32 PASSED [1.9339s] [ 49%] 2024-08-08T21:30:12.0281227Z test_decomp.py::TestDecompCUDA::test_comprehensive_remainder_cuda_int32 PASSED [0.1961s] [ 49%] 2024-08-08T21:30:12.0282732Z test_decomp.py::TestDecompCUDA::test_comprehensive_remainder_cuda_int64 PASSED [0.1959s] [ 49%] 2024-08-08T21:30:12.0284265Z test_decomp.py::TestDecompCUDA::test_comprehensive_renorm_cuda_complex64 PASSED [6.0765s] [ 49%] 2024-08-08T21:30:12.0285798Z test_decomp.py::TestDecompCUDA::test_comprehensive_repeat_cuda_bfloat16 PASSED [0.1454s] [ 49%] 2024-08-08T21:30:12.0287281Z test_decomp.py::TestDecompCUDA::test_comprehensive_repeat_cuda_bool PASSED [0.2550s] [ 50%] 2024-08-08T21:30:12.0288859Z test_decomp.py::TestDecompCUDA::test_comprehensive_repeat_interleave_cuda_float32 PASSED [0.2742s] [ 50%] 2024-08-08T21:30:12.0290522Z test_decomp.py::TestDecompCUDA::test_comprehensive_repeat_interleave_cuda_float64 PASSED [0.2680s] [ 50%] 2024-08-08T21:30:12.0292135Z test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_as_cuda_bfloat16 PASSED [0.0160s] [ 50%] 2024-08-08T21:30:12.0293700Z test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_as_cuda_complex64 PASSED [0.1838s] [ 50%] 2024-08-08T21:30:12.0295277Z test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_as_cuda_float32 PASSED [0.1119s] [ 50%] 2024-08-08T21:30:12.0296820Z test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_cuda_float64 PASSED [0.2410s] [ 50%] 2024-08-08T21:30:12.0298329Z test_decomp.py::TestDecompCUDA::test_comprehensive_reshape_cuda_int8 PASSED [0.0912s] [ 50%] 2024-08-08T21:30:12.0299823Z test_decomp.py::TestDecompCUDA::test_comprehensive_resize__cuda_float32 PASSED [0.0367s] [ 50%] 2024-08-08T21:30:12.0301465Z test_decomp.py::TestDecompCUDA::test_comprehensive_resize_as__cuda_complex128 PASSED [0.0573s] [ 50%] 2024-08-08T21:30:12.0303104Z test_decomp.py::TestDecompCUDA::test_comprehensive_resize_as__cuda_int64 PASSED [0.0255s] [ 50%] 2024-08-08T21:30:12.0304623Z test_decomp.py::TestDecompCUDA::test_comprehensive_resolve_conj_cuda_int8 PASSED [0.0058s] [ 50%] 2024-08-08T21:30:12.0306171Z test_decomp.py::TestDecompCUDA::test_comprehensive_resolve_neg_cuda_bool PASSED [0.0062s] [ 50%] 2024-08-08T21:30:12.0307729Z test_decomp.py::TestDecompCUDA::test_comprehensive_resolve_neg_cuda_float64 PASSED [0.0066s] [ 51%] 2024-08-08T21:30:12.0309258Z test_decomp.py::TestDecompCUDA::test_comprehensive_roll_cuda_float16 PASSED [0.2036s] [ 51%] 2024-08-08T21:30:12.0310722Z test_decomp.py::TestDecompCUDA::test_comprehensive_roll_cuda_float64 PASSED [1.6187s] [ 51%] 2024-08-08T21:30:12.0312224Z test_decomp.py::TestDecompCUDA::test_comprehensive_rot90_cuda_float64 PASSED [4.9078s] [ 51%] 2024-08-08T21:30:12.0313723Z test_decomp.py::TestDecompCUDA::test_comprehensive_round_cuda_float32 PASSED [0.1024s] [ 51%] 2024-08-08T21:30:12.0315209Z test_decomp.py::TestDecompCUDA::test_comprehensive_round_cuda_int64 PASSED [0.0400s] [ 51%] 2024-08-08T21:30:12.0316758Z test_decomp.py::TestDecompCUDA::test_comprehensive_round_decimals_0_cuda_float64 PASSED [0.0285s] [ 51%] 2024-08-08T21:30:12.0318388Z test_decomp.py::TestDecompCUDA::test_comprehensive_round_decimals_3_cuda_float32 PASSED [0.0299s] [ 51%] 2024-08-08T21:30:12.0320052Z test_decomp.py::TestDecompCUDA::test_comprehensive_round_decimals_neg_3_cuda_float16 PASSED [0.0140s] [ 51%] 2024-08-08T21:30:12.0321693Z test_decomp.py::TestDecompCUDA::test_comprehensive_rsqrt_cuda_complex128 PASSED [0.9118s] [ 51%] 2024-08-08T21:30:12.0323265Z test_decomp.py::TestDecompCUDA::test_comprehensive_rsub_cuda_float16 PASSED [0.0832s] [ 51%] 2024-08-08T21:30:12.0324802Z test_decomp.py::TestDecompCUDA::test_comprehensive_scalar_tensor_cuda_uint8 PASSED [0.0063s] [ 51%] 2024-08-08T21:30:12.0326389Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_add_cuda_bfloat16 PASSED [0.0181s] [ 52%] 2024-08-08T21:30:12.0327943Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_add_cuda_uint8 PASSED [0.0144s] [ 52%] 2024-08-08T21:30:12.0329463Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_cuda_int32 PASSED [0.0207s] [ 52%] 2024-08-08T21:30:12.0331039Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amax_cuda_int8 PASSED [0.0265s] [ 52%] 2024-08-08T21:30:12.0332688Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amax_cuda_uint8 PASSED [0.0254s] [ 52%] 2024-08-08T21:30:12.0334336Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amin_cuda_float32 PASSED [2.6741s] [ 52%] 2024-08-08T21:30:12.0336008Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amin_cuda_int16 PASSED [0.0259s] [ 52%] 2024-08-08T21:30:12.0337657Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_amin_cuda_uint8 PASSED [0.0255s] [ 52%] 2024-08-08T21:30:12.0339284Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_mean_cuda_int64 PASSED [0.0300s] [ 52%] 2024-08-08T21:30:12.0340937Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_sum_cuda_float64 PASSED [0.0407s] [ 52%] 2024-08-08T21:30:12.0342579Z test_decomp.py::TestDecompCUDA::test_comprehensive_scatter_reduce_sum_cuda_int64 PASSED [0.0255s] [ 52%] 2024-08-08T21:30:12.0344183Z test_decomp.py::TestDecompCUDA::test_comprehensive_searchsorted_cuda_int64 PASSED [0.2168s] [ 52%] 2024-08-08T21:30:12.0345790Z test_decomp.py::TestDecompCUDA::test_comprehensive_searchsorted_cuda_int8 PASSED [0.2115s] [ 53%] 2024-08-08T21:30:12.0347408Z test_decomp.py::TestDecompCUDA::test_comprehensive_searchsorted_cuda_uint8 PASSED [0.2119s] [ 53%] 2024-08-08T21:30:12.0348955Z test_decomp.py::TestDecompCUDA::test_comprehensive_select_cuda_complex32 PASSED [0.0079s] [ 53%] 2024-08-08T21:30:12.0350497Z test_decomp.py::TestDecompCUDA::test_comprehensive_select_cuda_complex64 PASSED [0.7363s] [ 53%] 2024-08-08T21:30:12.0351993Z test_decomp.py::TestDecompCUDA::test_comprehensive_select_cuda_int32 PASSED [0.0079s] [ 53%] 2024-08-08T21:30:12.0353472Z test_decomp.py::TestDecompCUDA::test_comprehensive_select_cuda_int64 PASSED [0.0078s] [ 53%] 2024-08-08T21:30:12.0355069Z test_decomp.py::TestDecompCUDA::test_comprehensive_select_scatter_cuda_int8 PASSED [0.3049s] [ 53%] 2024-08-08T21:30:12.0356617Z test_decomp.py::TestDecompCUDA::test_comprehensive_sgn_cuda_complex32 PASSED [0.8296s] [ 53%] 2024-08-08T21:30:12.0358119Z test_decomp.py::TestDecompCUDA::test_comprehensive_sigmoid_cuda_float16 PASSED [0.0396s] [ 53%] 2024-08-08T21:30:12.0360202Z test_decomp.py::TestDecompCUDA::test_comprehensive_signal_windows_bartlett_cuda_float32 PASSED [0.4008s] [ 53%] 2024-08-08T21:30:12.0361973Z test_decomp.py::TestDecompCUDA::test_comprehensive_signal_windows_general_hamming_cuda_float64 PASSED [0.7338s] [ 53%] 2024-08-08T21:30:12.0363693Z test_decomp.py::TestDecompCUDA::test_comprehensive_signal_windows_hann_cuda_float32 PASSED [0.7493s] [ 53%] 2024-08-08T21:30:12.0365375Z test_decomp.py::TestDecompCUDA::test_comprehensive_signal_windows_kaiser_cuda_float32 PASSED [1.2029s] [ 53%] 2024-08-08T21:30:12.0367064Z test_decomp.py::TestDecompCUDA::test_comprehensive_signbit_cuda_bool PASSED [0.0404s] [ 54%] 2024-08-08T21:30:12.0368661Z test_decomp.py::TestDecompCUDA::test_comprehensive_signbit_cuda_int32 PASSED [0.0402s] [ 54%] 2024-08-08T21:30:12.0370141Z test_decomp.py::TestDecompCUDA::test_comprehensive_signbit_cuda_uint8 PASSED [0.0402s] [ 54%] 2024-08-08T21:30:12.0371638Z test_decomp.py::TestDecompCUDA::test_comprehensive_sinc_cuda_float32 PASSED [0.3113s] [ 54%] 2024-08-08T21:30:12.0373118Z test_decomp.py::TestDecompCUDA::test_comprehensive_sinc_cuda_uint8 PASSED [0.3643s] [ 54%] 2024-08-08T21:30:12.0374602Z test_decomp.py::TestDecompCUDA::test_comprehensive_sinh_cuda_bfloat16 PASSED [0.0101s] [ 54%] 2024-08-08T21:30:12.0376104Z test_decomp.py::TestDecompCUDA::test_comprehensive_sinh_cuda_complex128 PASSED [1.2194s] [ 54%] 2024-08-08T21:30:12.0377616Z test_decomp.py::TestDecompCUDA::test_comprehensive_sinh_cuda_float32 PASSED [0.2037s] [ 54%] 2024-08-08T21:30:12.0379130Z test_decomp.py::TestDecompCUDA::test_comprehensive_slice_cuda_complex64 PASSED [4.5494s] [ 54%] 2024-08-08T21:30:12.0380629Z test_decomp.py::TestDecompCUDA::test_comprehensive_slice_cuda_float32 PASSED [3.2043s] [ 54%] 2024-08-08T21:30:12.0382190Z test_decomp.py::TestDecompCUDA::test_comprehensive_slice_scatter_cuda_float64 PASSED [10.2843s] [ 54%] 2024-08-08T21:30:12.0383775Z test_decomp.py::TestDecompCUDA::test_comprehensive_slice_scatter_cuda_int32 PASSED [4.6092s] [ 54%] 2024-08-08T21:30:12.0385376Z test_decomp.py::TestDecompCUDA::test_comprehensive_softmax_with_dtype_cuda_int8 PASSED [0.7945s] [ 55%] 2024-08-08T21:30:12.0386907Z test_decomp.py::TestDecompCUDA::test_comprehensive_sort_cuda_float64 PASSED [0.8152s] [ 55%] 2024-08-08T21:30:12.0388630Z test_decomp.py::TestDecompCUDA::test_comprehensive_sparse_mm_reduce_cuda_bfloat16 SKIPPED [0.0013s] (Only runs on cpu) [ 55%] 2024-08-08T21:30:12.0390470Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_airy_ai_cuda_float32 PASSED [0.5757s] [ 55%] 2024-08-08T21:30:12.0392163Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_airy_ai_cuda_int32 PASSED [0.4593s] [ 55%] 2024-08-08T21:30:12.0393750Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_j0_cuda_int16 PASSED [0.7647s] [ 55%] 2024-08-08T21:30:12.0395379Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_j0_cuda_int32 PASSED [0.0129s] [ 55%] 2024-08-08T21:30:12.0396997Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_j0_cuda_int8 PASSED [0.0133s] [ 55%] 2024-08-08T21:30:12.0398614Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_j1_cuda_float64 PASSED [0.1577s] [ 55%] 2024-08-08T21:30:12.0400250Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_bessel_y0_cuda_int64 PASSED [0.5487s] [ 55%] 2024-08-08T21:30:12.0401969Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_chebyshev_polynomial_t_cuda_int32 PASSED [1.5075s] [ 55%] 2024-08-08T21:30:12.0403756Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_chebyshev_polynomial_t_cuda_int64 PASSED [0.0110s] [ 55%] 2024-08-08T21:30:12.0405935Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_chebyshev_polynomial_v_cuda_bool SKIPPED [0.0003s] (Skipping - testing takes an unreasonably long time, #79528) [ 56%] 2024-08-08T21:30:12.0408550Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_chebyshev_polynomial_v_cuda_int64 SKIPPED [0.0002s] (Skipping - testing takes an unreasonably long time, #79528) [ 56%] 2024-08-08T21:30:12.0410598Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_entr_cuda_float32 PASSED [0.2283s] [ 56%] 2024-08-08T21:30:12.0412233Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_entr_cuda_int8 PASSED [0.3656s] [ 56%] 2024-08-08T21:30:12.0413845Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_erfcx_cuda_bool PASSED [1.2107s] [ 56%] 2024-08-08T21:30:12.0415528Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_hermite_polynomial_h_cuda_bool PASSED [1.3006s] [ 56%] 2024-08-08T21:30:12.0417309Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_hermite_polynomial_he_cuda_float64 PASSED [0.5962s] [ 56%] 2024-08-08T21:30:12.0418969Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_i0e_cuda_int8 PASSED [0.2365s] [ 56%] 2024-08-08T21:30:12.0420498Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_i0e_cuda_uint8 PASSED [0.0099s] [ 56%] 2024-08-08T21:30:12.0422040Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_i1_cuda_bool PASSED [0.2475s] [ 56%] 2024-08-08T21:30:12.0423600Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_i1_cuda_float32 PASSED [0.2259s] [ 56%] 2024-08-08T21:30:12.0425153Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_i1_cuda_int32 PASSED [0.0099s] [ 56%] 2024-08-08T21:30:12.0426709Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_log_ndtr_cuda_int32 PASSED [0.4962s] [ 57%] 2024-08-08T21:30:12.0428384Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_modified_bessel_k0_cuda_int32 PASSED [0.3240s] [ 57%] 2024-08-08T21:30:12.0430381Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_ndtr_cuda_float16 SKIPPED [0.0028s] (special.ndtr in torch.float16 not supported) [ 57%] 2024-08-08T21:30:12.0432251Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_ndtri_cuda_bool PASSED [0.2805s] [ 57%] 2024-08-08T21:30:12.0433960Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_scaled_modified_bessel_k1_cuda_float64 PASSED [0.4527s] [ 57%] 2024-08-08T21:30:12.0435848Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_scaled_modified_bessel_k1_cuda_int16 PASSED [0.3093s] [ 57%] 2024-08-08T21:30:12.0438219Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_t_cuda_float64 SKIPPED [0.0003s] (Skipping - testing takes an unreasonably long time, #79528) [ 57%] 2024-08-08T21:30:12.0440892Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_u_cuda_int16 SKIPPED [0.0002s] (Skipping - testing takes an unreasonably long time, #79528) [ 57%] 2024-08-08T21:30:12.0443526Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_u_cuda_int32 SKIPPED [0.0002s] (Skipping - testing takes an unreasonably long time, #79528) [ 57%] 2024-08-08T21:30:12.0446197Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_w_cuda_float64 SKIPPED [0.0002s] (Skipping - testing takes an unreasonably long time, #79528) [ 57%] 2024-08-08T21:30:12.0448861Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_shifted_chebyshev_polynomial_w_cuda_int16 SKIPPED [0.0002s] (Skipping - testing takes an unreasonably long time, #79528) [ 57%] 2024-08-08T21:30:12.0451037Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_spherical_bessel_j0_cuda_float64 PASSED [0.1996s] [ 57%] 2024-08-08T21:30:12.0452775Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_spherical_bessel_j0_cuda_int8 PASSED [0.4556s] [ 57%] 2024-08-08T21:30:12.0454484Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_xlog1py_cuda_int16 PASSED [2.4539s] [ 58%] 2024-08-08T21:30:12.0456144Z test_decomp.py::TestDecompCUDA::test_comprehensive_special_xlog1py_cuda_int32 PASSED [2.4572s] [ 58%] 2024-08-08T21:30:12.0457755Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_cuda_float32 PASSED [0.2718s] [ 58%] 2024-08-08T21:30:12.0459646Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_cuda_int32 PASSED [0.1301s] [ 58%] 2024-08-08T21:30:12.0461129Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_cuda_int64 PASSED [0.1293s] [ 58%] 2024-08-08T21:30:12.0462683Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_list_args_cuda_float16 PASSED [0.0205s] [ 58%] 2024-08-08T21:30:12.0464307Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_list_args_cuda_float64 PASSED [0.3566s] [ 58%] 2024-08-08T21:30:12.0465900Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_list_args_cuda_uint8 PASSED [0.1458s] [ 58%] 2024-08-08T21:30:12.0467575Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_copy_cuda_complex32 PASSED [0.6437s] [ 58%] 2024-08-08T21:30:12.0469289Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_copy_cuda_float64 PASSED [0.5944s] [ 58%] 2024-08-08T21:30:12.0470945Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_copy_cuda_int16 PASSED [0.2765s] [ 58%] 2024-08-08T21:30:12.0472611Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_copy_cuda_int32 PASSED [0.2765s] [ 58%] 2024-08-08T21:30:12.0474267Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_cuda_complex128 PASSED [0.7951s] [ 59%] 2024-08-08T21:30:12.0475937Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_cuda_complex32 PASSED [0.9740s] [ 59%] 2024-08-08T21:30:12.0477548Z test_decomp.py::TestDecompCUDA::test_comprehensive_split_with_sizes_cuda_uint8 PASSED [0.1924s] [ 59%] 2024-08-08T21:30:12.0479101Z test_decomp.py::TestDecompCUDA::test_comprehensive_sqrt_cuda_bfloat16 PASSED [0.0101s] [ 59%] 2024-08-08T21:30:12.0480717Z test_decomp.py::TestDecompCUDA::test_comprehensive_sqrt_cuda_complex32 PASSED [0.9885s] [ 59%] 2024-08-08T21:30:12.0482283Z test_decomp.py::TestDecompCUDA::test_comprehensive_sqrt_cuda_int32 PASSED [0.0412s] [ 59%] 2024-08-08T21:30:12.0483748Z test_decomp.py::TestDecompCUDA::test_comprehensive_square_cuda_float32 PASSED [0.0877s] [ 59%] 2024-08-08T21:30:12.0485254Z test_decomp.py::TestDecompCUDA::test_comprehensive_square_cuda_int32 PASSED [0.0187s] [ 59%] 2024-08-08T21:30:12.0486784Z test_decomp.py::TestDecompCUDA::test_comprehensive_squeeze_cuda_complex128 PASSED [0.2976s] [ 59%] 2024-08-08T21:30:12.0488304Z test_decomp.py::TestDecompCUDA::test_comprehensive_squeeze_cuda_float16 PASSED [0.0326s] [ 59%] 2024-08-08T21:30:12.0489820Z test_decomp.py::TestDecompCUDA::test_comprehensive_squeeze_cuda_int32 PASSED [0.1008s] [ 59%] 2024-08-08T21:30:12.0491414Z test_decomp.py::TestDecompCUDA::test_comprehensive_squeeze_multiple_cuda_complex32 PASSED [0.0877s] [ 59%] 2024-08-08T21:30:12.0493007Z test_decomp.py::TestDecompCUDA::test_comprehensive_stack_cuda_complex32 PASSED [0.3968s] [ 60%] 2024-08-08T21:30:12.0494499Z test_decomp.py::TestDecompCUDA::test_comprehensive_std_cuda_bfloat16 PASSED [0.3711s] [ 60%] 2024-08-08T21:30:12.0496003Z test_decomp.py::TestDecompCUDA::test_comprehensive_std_cuda_complex128 PASSED [3.5380s] [ 60%] 2024-08-08T21:30:12.0497522Z test_decomp.py::TestDecompCUDA::test_comprehensive_std_mean_cuda_float64 PASSED [3.9079s] [ 60%] 2024-08-08T21:30:12.0499144Z test_decomp.py::TestDecompCUDA::test_comprehensive_std_mean_unbiased_cuda_complex128 PASSED [0.4448s] [ 60%] 2024-08-08T21:30:12.0500849Z test_decomp.py::TestDecompCUDA::test_comprehensive_std_unbiased_cuda_complex128 PASSED [0.2973s] [ 60%] 2024-08-08T21:30:12.0502492Z test_decomp.py::TestDecompCUDA::test_comprehensive_sub_cuda_complex32 PASSED [0.5353s] [ 60%] 2024-08-08T21:30:12.0503971Z test_decomp.py::TestDecompCUDA::test_comprehensive_sub_cuda_uint8 PASSED [0.2082s] [ 60%] 2024-08-08T21:30:12.0505400Z test_decomp.py::TestDecompCUDA::test_comprehensive_sum_cuda_int32 PASSED [0.0802s] [ 60%] 2024-08-08T21:30:12.0506896Z test_decomp.py::TestDecompCUDA::test_comprehensive_sum_to_size_cuda_bool PASSED [0.0982s] [ 60%] 2024-08-08T21:30:12.0508420Z test_decomp.py::TestDecompCUDA::test_comprehensive_svd_cuda_complex64 PASSED [94.1913s] [ 60%] 2024-08-08T21:30:12.0509950Z test_decomp.py::TestDecompCUDA::test_comprehensive_t_copy_cuda_complex64 PASSED [0.1643s] [ 60%] 2024-08-08T21:30:12.0511440Z test_decomp.py::TestDecompCUDA::test_comprehensive_t_copy_cuda_int8 PASSED [0.0262s] [ 60%] 2024-08-08T21:30:12.0512894Z test_decomp.py::TestDecompCUDA::test_comprehensive_t_cuda_bool PASSED [0.0180s] [ 61%] 2024-08-08T21:30:12.0514333Z test_decomp.py::TestDecompCUDA::test_comprehensive_t_cuda_float16 PASSED [0.0205s] [ 61%] 2024-08-08T21:30:12.0515775Z test_decomp.py::TestDecompCUDA::test_comprehensive_t_cuda_int8 PASSED [0.0183s] [ 61%] 2024-08-08T21:30:12.0517284Z test_decomp.py::TestDecompCUDA::test_comprehensive_take_along_dim_cuda_float64 PASSED [0.1072s] [ 61%] 2024-08-08T21:30:12.0518882Z test_decomp.py::TestDecompCUDA::test_comprehensive_take_along_dim_cuda_uint8 PASSED [0.0359s] [ 61%] 2024-08-08T21:30:12.0520391Z test_decomp.py::TestDecompCUDA::test_comprehensive_take_cuda_bool PASSED [0.0356s] [ 61%] 2024-08-08T21:30:12.0521853Z test_decomp.py::TestDecompCUDA::test_comprehensive_tan_cuda_bfloat16 PASSED [0.0134s] [ 61%] 2024-08-08T21:30:12.0523339Z test_decomp.py::TestDecompCUDA::test_comprehensive_tan_cuda_float16 PASSED [0.0123s] [ 61%] 2024-08-08T21:30:12.0524917Z test_decomp.py::TestDecompCUDA::test_comprehensive_tensor_split_cuda_bool PASSED [0.5324s] [ 61%] 2024-08-08T21:30:12.0526473Z test_decomp.py::TestDecompCUDA::test_comprehensive_tile_cuda_int64 PASSED [0.3232s] [ 61%] 2024-08-08T21:30:12.0527937Z test_decomp.py::TestDecompCUDA::test_comprehensive_to_cuda_complex128 PASSED [9.8778s] [ 61%] 2024-08-08T21:30:12.0529420Z test_decomp.py::TestDecompCUDA::test_comprehensive_to_cuda_float32 PASSED [8.0127s] [ 61%] 2024-08-08T21:30:12.0530884Z test_decomp.py::TestDecompCUDA::test_comprehensive_to_cuda_float64 PASSED [4.4337s] [ 62%] 2024-08-08T21:30:12.0532317Z test_decomp.py::TestDecompCUDA::test_comprehensive_to_cuda_int64 PASSED [6.2945s] [ 62%] 2024-08-08T21:30:12.0533809Z test_decomp.py::TestDecompCUDA::test_comprehensive_to_sparse_cuda_uint8 PASSED [0.0153s] [ 62%] 2024-08-08T21:30:12.0535308Z test_decomp.py::TestDecompCUDA::test_comprehensive_topk_cuda_int32 PASSED [0.0204s] [ 62%] 2024-08-08T21:30:12.0536766Z test_decomp.py::TestDecompCUDA::test_comprehensive_topk_cuda_int8 PASSED [0.0145s] [ 62%] 2024-08-08T21:30:12.0538473Z test_decomp.py::TestDecompCUDA::test_comprehensive_torch_ops_aten__efficient_attention_forward_cuda_float16 PASSED [0.0855s] [ 62%] 2024-08-08T21:30:12.0540370Z test_decomp.py::TestDecompCUDA::test_comprehensive_torch_ops_aten__safe_softmax_default_cuda_int16 PASSED [0.5617s] [ 62%] 2024-08-08T21:30:12.0542041Z test_decomp.py::TestDecompCUDA::test_comprehensive_trace_cuda_complex64 PASSED [0.0814s] [ 62%] 2024-08-08T21:30:12.0543558Z test_decomp.py::TestDecompCUDA::test_comprehensive_trace_cuda_float16 PASSED [0.0156s] [ 62%] 2024-08-08T21:30:12.0545085Z test_decomp.py::TestDecompCUDA::test_comprehensive_trace_cuda_int32 PASSED [0.0131s] [ 62%] 2024-08-08T21:30:12.0546647Z test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_bool PASSED [0.0841s] [ 62%] 2024-08-08T21:30:12.0548195Z test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_float32 PASSED [0.2794s] [ 62%] 2024-08-08T21:30:12.0549729Z test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_float64 PASSED [0.2797s] [ 63%] 2024-08-08T21:30:12.0551274Z test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_int32 PASSED [0.0841s] [ 63%] 2024-08-08T21:30:12.0552803Z test_decomp.py::TestDecompCUDA::test_comprehensive_transpose_cuda_int8 PASSED [0.0846s] [ 63%] 2024-08-08T21:30:12.0554374Z test_decomp.py::TestDecompCUDA::test_comprehensive_trapezoid_cuda_int16 PASSED [0.1966s] [ 63%] 2024-08-08T21:30:12.0555875Z test_decomp.py::TestDecompCUDA::test_comprehensive_trapz_cuda_float32 PASSED [5.3967s] [ 63%] 2024-08-08T21:30:12.0557369Z test_decomp.py::TestDecompCUDA::test_comprehensive_trapz_cuda_int8 PASSED [0.1960s] [ 63%] 2024-08-08T21:30:12.0559365Z test_decomp.py::TestDecompCUDA::test_comprehensive_triangular_solve_cuda_complex64 PASSED [1.4331s] [ 63%] 2024-08-08T21:30:12.0560953Z test_decomp.py::TestDecompCUDA::test_comprehensive_tril_cuda_bool PASSED [0.5638s] [ 63%] 2024-08-08T21:30:12.0562440Z test_decomp.py::TestDecompCUDA::test_comprehensive_tril_cuda_complex128 PASSED [2.7575s] [ 63%] 2024-08-08T21:30:12.0563955Z test_decomp.py::TestDecompCUDA::test_comprehensive_tril_cuda_complex32 PASSED [1.3851s] [ 63%] 2024-08-08T21:30:12.0565455Z test_decomp.py::TestDecompCUDA::test_comprehensive_triu_cuda_float64 PASSED [1.9930s] [ 63%] 2024-08-08T21:30:12.0566908Z test_decomp.py::TestDecompCUDA::test_comprehensive_triu_cuda_int64 PASSED [0.8545s] [ 63%] 2024-08-08T21:30:12.0568471Z test_decomp.py::TestDecompCUDA::test_comprehensive_triu_cuda_uint8 PASSED [0.8465s] [ 64%] 2024-08-08T21:30:12.0570010Z test_decomp.py::TestDecompCUDA::test_comprehensive_trunc_cuda_uint8 PASSED [0.0410s] [ 64%] 2024-08-08T21:30:12.0571508Z test_decomp.py::TestDecompCUDA::test_comprehensive_unbind_cuda_float64 PASSED [0.8333s] [ 64%] 2024-08-08T21:30:12.0572992Z test_decomp.py::TestDecompCUDA::test_comprehensive_unbind_cuda_int64 PASSED [0.2852s] [ 64%] 2024-08-08T21:30:12.0574549Z test_decomp.py::TestDecompCUDA::test_comprehensive_unfold_copy_cuda_complex32 PASSED [9.5455s] [ 64%] 2024-08-08T21:30:12.0576126Z test_decomp.py::TestDecompCUDA::test_comprehensive_unfold_copy_cuda_int32 PASSED [4.6250s] [ 64%] 2024-08-08T21:30:12.0577671Z test_decomp.py::TestDecompCUDA::test_comprehensive_uniform_cuda_float64 PASSED [0.0087s] [ 64%] 2024-08-08T21:30:12.0579249Z test_decomp.py::TestDecompCUDA::test_comprehensive_unique_consecutive_cuda_int32 PASSED [0.2827s] [ 64%] 2024-08-08T21:30:12.0580821Z test_decomp.py::TestDecompCUDA::test_comprehensive_unique_cuda_uint16 PASSED [0.5978s] [ 64%] 2024-08-08T21:30:12.0582358Z test_decomp.py::TestDecompCUDA::test_comprehensive_unravel_index_cuda_int8 PASSED [1.0785s] [ 64%] 2024-08-08T21:30:12.0583929Z test_decomp.py::TestDecompCUDA::test_comprehensive_unsafe_chunk_cuda_bfloat16 PASSED [0.0346s] [ 64%] 2024-08-08T21:30:12.0585523Z test_decomp.py::TestDecompCUDA::test_comprehensive_unsafe_chunk_cuda_int32 PASSED [0.5813s] [ 64%] 2024-08-08T21:30:12.0587076Z test_decomp.py::TestDecompCUDA::test_comprehensive_unsafe_split_cuda_bool PASSED [0.1956s] [ 64%] 2024-08-08T21:30:12.0588652Z test_decomp.py::TestDecompCUDA::test_comprehensive_unsqueeze_copy_cuda_int64 PASSED [0.1728s] [ 65%] 2024-08-08T21:30:12.0590281Z test_decomp.py::TestDecompCUDA::test_comprehensive_unsqueeze_cuda_float64 PASSED [0.2843s] [ 65%] 2024-08-08T21:30:12.0591902Z test_decomp.py::TestDecompCUDA::test_comprehensive_unsqueeze_cuda_int32 PASSED [0.0576s] [ 65%] 2024-08-08T21:30:12.0593426Z test_decomp.py::TestDecompCUDA::test_comprehensive_unsqueeze_cuda_int8 PASSED [0.0590s] [ 65%] 2024-08-08T21:30:12.0594928Z test_decomp.py::TestDecompCUDA::test_comprehensive_var_cuda_bfloat16 PASSED [0.1211s] [ 65%] 2024-08-08T21:30:12.0596439Z test_decomp.py::TestDecompCUDA::test_comprehensive_var_mean_cuda_complex64 PASSED [3.2224s] [ 65%] 2024-08-08T21:30:12.0598064Z test_decomp.py::TestDecompCUDA::test_comprehensive_var_mean_unbiased_cuda_bfloat16 PASSED [0.0346s] [ 65%] 2024-08-08T21:30:12.0599721Z test_decomp.py::TestDecompCUDA::test_comprehensive_var_mean_unbiased_cuda_float32 PASSED [0.1394s] [ 65%] 2024-08-08T21:30:12.0601278Z test_decomp.py::TestDecompCUDA::test_comprehensive_vdot_cuda_bfloat16 PASSED [0.0156s] [ 65%] 2024-08-08T21:30:12.0602790Z test_decomp.py::TestDecompCUDA::test_comprehensive_vdot_cuda_float64 PASSED [0.0563s] [ 65%] 2024-08-08T21:30:12.0604305Z test_decomp.py::TestDecompCUDA::test_comprehensive_view_as_cuda_float16 PASSED [0.0160s] [ 65%] 2024-08-08T21:30:12.0605841Z test_decomp.py::TestDecompCUDA::test_comprehensive_view_copy_cuda_float64 PASSED [0.4669s] [ 65%] 2024-08-08T21:30:12.0607342Z test_decomp.py::TestDecompCUDA::test_comprehensive_view_cuda_float16 PASSED [0.0245s] [ 66%] 2024-08-08T21:30:12.0608864Z test_decomp.py::TestDecompCUDA::test_comprehensive_vsplit_cuda_complex64 PASSED [2.7636s] [ 66%] 2024-08-08T21:30:12.0610375Z test_decomp.py::TestDecompCUDA::test_comprehensive_vstack_cuda_int64 PASSED [0.0401s] [ 66%] 2024-08-08T21:30:12.0611892Z test_decomp.py::TestDecompCUDA::test_comprehensive_where_cuda_complex32 PASSED [0.4885s] [ 66%] 2024-08-08T21:30:12.0613475Z test_decomp.py::TestDecompCUDA::test_comprehensive_where_cuda_complex64 PASSED [2.3642s] [ 66%] 2024-08-08T21:30:12.0615045Z test_decomp.py::TestDecompCUDA::test_comprehensive_where_cuda_float16 PASSED [0.0881s] [ 66%] 2024-08-08T21:30:12.0616528Z test_decomp.py::TestDecompCUDA::test_comprehensive_where_cuda_int32 PASSED [0.2514s] [ 66%] 2024-08-08T21:30:12.0617981Z test_decomp.py::TestDecompCUDA::test_comprehensive_xlogy_cuda_int8 PASSED [2.5081s] [ 66%] 2024-08-08T21:30:12.0619747Z test_decomp.py::TestDecompCUDA::test_comprehensive_zero__cuda_float64 SKIPPED [0.0028s] (zero_ in torch.float64 not supported) [ 66%] 2024-08-08T21:30:12.0621501Z test_decomp.py::TestDecompCUDA::test_comprehensive_zeros_cuda_complex128 PASSED [0.0293s] [ 66%] 2024-08-08T21:30:12.0623063Z test_decomp.py::TestDecompCUDA::test_comprehensive_zeros_like_cuda_float16 PASSED [0.0166s] [ 66%] 2024-08-08T21:30:12.0624600Z test_decomp.py::TestDecompCUDA::test_comprehensive_zeros_like_cuda_int32 PASSED [0.0249s] [ 66%] 2024-08-08T21:30:12.0626044Z test_decomp.py::TestDecompCUDA::test_masked_fill_cuda PASSED [0.0138s] [ 67%] 2024-08-08T21:30:12.0627543Z test_decomp.py::TestDecompCUDA::test_quick__batch_norm_with_update_cuda_bfloat16 PASSED [0.0389s] [ 67%] 2024-08-08T21:30:12.0629091Z test_decomp.py::TestDecompCUDA::test_quick__chunk_cat_cuda_complex128 PASSED [0.3666s] [ 67%] 2024-08-08T21:30:12.0630549Z test_decomp.py::TestDecompCUDA::test_quick__chunk_cat_cuda_float32 PASSED [0.2071s] [ 67%] 2024-08-08T21:30:12.0631988Z test_decomp.py::TestDecompCUDA::test_quick__chunk_cat_cuda_int8 PASSED [0.1357s] [ 67%] 2024-08-08T21:30:12.0633473Z test_decomp.py::TestDecompCUDA::test_quick__chunk_cat_cuda_uint8 PASSED [0.1355s] [ 67%] 2024-08-08T21:30:12.0635042Z test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_cuda_complex64 PASSED [0.3900s] [ 67%] 2024-08-08T21:30:12.0636623Z test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_cuda_int16 PASSED [0.3131s] [ 67%] 2024-08-08T21:30:12.0638173Z test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_cuda_int32 PASSED [0.3132s] [ 67%] 2024-08-08T21:30:12.0639845Z test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_put_accumulate_cuda_complex64 PASSED [0.9465s] [ 67%] 2024-08-08T21:30:12.0641600Z test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_put_accumulate_cuda_float32 PASSED [0.6329s] [ 67%] 2024-08-08T21:30:12.0643349Z test_decomp.py::TestDecompCUDA::test_quick__unsafe_masked_index_put_accumulate_cuda_uint8 PASSED [0.5344s] [ 67%] 2024-08-08T21:30:12.0645420Z test_decomp.py::TestDecompCUDA::test_quick__upsample_bilinear2d_aa_cuda_float64 SKIPPED [0.0028s] (_upsample_bilinear2d_aa in torch.float64 not supported) [ 67%] 2024-08-08T21:30:12.0647258Z test_decomp.py::TestDecompCUDA::test_quick_acos_cuda_float32 PASSED [0.0201s] [ 68%] 2024-08-08T21:30:12.0648626Z test_decomp.py::TestDecompCUDA::test_quick_acos_cuda_float64 PASSED [0.0178s] [ 68%] 2024-08-08T21:30:12.0649999Z test_decomp.py::TestDecompCUDA::test_quick_acos_cuda_uint8 PASSED [0.0130s] [ 68%] 2024-08-08T21:30:12.0651350Z test_decomp.py::TestDecompCUDA::test_quick_add_cuda_uint8 PASSED [0.1723s] [ 68%] 2024-08-08T21:30:12.0652743Z test_decomp.py::TestDecompCUDA::test_quick_addcdiv_cuda_complex128 PASSED [1.8398s] [ 68%] 2024-08-08T21:30:12.0654237Z test_decomp.py::TestDecompCUDA::test_quick_addcmul_cuda_float32 PASSED [0.2034s] [ 68%] 2024-08-08T21:30:12.0672900Z test_decomp.py::TestDecompCUDA::test_quick_addcmul_cuda_int16 PASSED [0.1079s] [ 68%] 2024-08-08T21:30:12.0674881Z test_decomp.py::TestDecompCUDA::test_quick_addmm_decomposed_cuda_complex128 PASSED [0.1377s] [ 68%] 2024-08-08T21:30:12.0676561Z test_decomp.py::TestDecompCUDA::test_quick_addmv_cuda_float32 PASSED [0.1098s] [ 68%] 2024-08-08T21:30:12.0678030Z test_decomp.py::TestDecompCUDA::test_quick_addr_cuda_bool PASSED [0.0315s] [ 68%] 2024-08-08T21:30:12.0679538Z test_decomp.py::TestDecompCUDA::test_quick_alias_copy_cuda_float16 PASSED [0.0084s] [ 68%] 2024-08-08T21:30:12.0681040Z test_decomp.py::TestDecompCUDA::test_quick_amin_cuda_bfloat16 PASSED [0.0444s] [ 68%] 2024-08-08T21:30:12.0682510Z test_decomp.py::TestDecompCUDA::test_quick_amin_cuda_uint8 PASSED [0.0700s] [ 69%] 2024-08-08T21:30:12.0683985Z test_decomp.py::TestDecompCUDA::test_quick_aminmax_cuda_bool PASSED [0.1127s] [ 69%] 2024-08-08T21:30:12.0685443Z test_decomp.py::TestDecompCUDA::test_quick_any_cuda_uint8 PASSED [0.0739s] [ 69%] 2024-08-08T21:30:12.0686902Z test_decomp.py::TestDecompCUDA::test_quick_arange_cuda_float64 PASSED [0.0511s] [ 69%] 2024-08-08T21:30:12.0688396Z test_decomp.py::TestDecompCUDA::test_quick_arange_cuda_int16 PASSED [0.0378s] [ 69%] 2024-08-08T21:30:12.0689974Z test_decomp.py::TestDecompCUDA::test_quick_as_strided_scatter_cuda_bfloat16 PASSED [0.0196s] [ 69%] 2024-08-08T21:30:12.0691637Z test_decomp.py::TestDecompCUDA::test_quick_as_strided_scatter_cuda_float32 PASSED [0.0719s] [ 69%] 2024-08-08T21:30:12.0693215Z test_decomp.py::TestDecompCUDA::test_quick_asin_cuda_complex128 PASSED [0.0984s] [ 69%] 2024-08-08T21:30:12.0694697Z test_decomp.py::TestDecompCUDA::test_quick_asin_cuda_int16 PASSED [0.0413s] [ 69%] 2024-08-08T21:30:12.0696199Z test_decomp.py::TestDecompCUDA::test_quick_asin_cuda_int64 PASSED [0.0420s] [ 69%] 2024-08-08T21:30:12.0697707Z test_decomp.py::TestDecompCUDA::test_quick_asin_cuda_uint8 PASSED [0.0411s] [ 69%] 2024-08-08T21:30:12.0699200Z test_decomp.py::TestDecompCUDA::test_quick_asinh_cuda_complex64 PASSED [0.1041s] [ 69%] 2024-08-08T21:30:12.0700706Z test_decomp.py::TestDecompCUDA::test_quick_asinh_cuda_float32 PASSED [0.0540s] [ 70%] 2024-08-08T21:30:12.0702162Z test_decomp.py::TestDecompCUDA::test_quick_asinh_cuda_uint8 PASSED [0.0422s] [ 70%] 2024-08-08T21:30:12.0703645Z test_decomp.py::TestDecompCUDA::test_quick_atan2_cuda_float32 PASSED [0.2202s] [ 70%] 2024-08-08T21:30:12.0705114Z test_decomp.py::TestDecompCUDA::test_quick_atan2_cuda_int16 PASSED [0.1594s] [ 70%] 2024-08-08T21:30:12.0706569Z test_decomp.py::TestDecompCUDA::test_quick_atan2_cuda_uint8 PASSED [0.1590s] [ 70%] 2024-08-08T21:30:12.0708082Z test_decomp.py::TestDecompCUDA::test_quick_atan_cuda_int32 PASSED [0.0430s] [ 70%] 2024-08-08T21:30:12.0709577Z test_decomp.py::TestDecompCUDA::test_quick_atanh_cuda_float16 PASSED [0.0068s] [ 70%] 2024-08-08T21:30:12.0711062Z test_decomp.py::TestDecompCUDA::test_quick_atanh_cuda_int32 PASSED [0.0417s] [ 70%] 2024-08-08T21:30:12.0712498Z test_decomp.py::TestDecompCUDA::test_quick_atanh_cuda_uint8 PASSED [0.0412s] [ 70%] 2024-08-08T21:30:12.0713993Z test_decomp.py::TestDecompCUDA::test_quick_bitwise_not_cuda_bool PASSED [0.0116s] [ 70%] 2024-08-08T21:30:12.0715524Z test_decomp.py::TestDecompCUDA::test_quick_bitwise_not_cuda_int32 PASSED [0.0119s] [ 70%] 2024-08-08T21:30:12.0717058Z test_decomp.py::TestDecompCUDA::test_quick_bitwise_not_cuda_int8 PASSED [0.0116s] [ 70%] 2024-08-08T21:30:12.0718573Z test_decomp.py::TestDecompCUDA::test_quick_bitwise_or_cuda_int8 PASSED [0.1544s] [ 71%] 2024-08-08T21:30:12.0720161Z test_decomp.py::TestDecompCUDA::test_quick_bitwise_right_shift_cuda_uint8 PASSED [0.1529s] [ 71%] 2024-08-08T21:30:12.0721825Z test_decomp.py::TestDecompCUDA::test_quick_bitwise_xor_cuda_int16 PASSED [0.1538s] [ 71%] 2024-08-08T21:30:12.0723413Z test_decomp.py::TestDecompCUDA::test_quick_block_diag_cuda_int32 PASSED [0.0289s] [ 71%] 2024-08-08T21:30:12.0724940Z test_decomp.py::TestDecompCUDA::test_quick_bucketize_cuda_bfloat16 PASSED [0.1874s] [ 71%] 2024-08-08T21:30:12.0726443Z test_decomp.py::TestDecompCUDA::test_quick_cat_cuda_float16 PASSED [0.0222s] [ 71%] 2024-08-08T21:30:12.0727880Z test_decomp.py::TestDecompCUDA::test_quick_cat_cuda_int8 PASSED [0.0594s] [ 71%] 2024-08-08T21:30:12.0729317Z test_decomp.py::TestDecompCUDA::test_quick_cauchy_cuda_float16 XFAIL [0.0081s] [ 71%] 2024-08-08T21:30:12.0730811Z test_decomp.py::TestDecompCUDA::test_quick_ceil_cuda_float16 PASSED [0.0071s] [ 71%] 2024-08-08T21:30:12.0732312Z test_decomp.py::TestDecompCUDA::test_quick_clamp_max_cuda_int32 PASSED [0.1508s] [ 71%] 2024-08-08T21:30:12.0733849Z test_decomp.py::TestDecompCUDA::test_quick_clamp_min_cuda_float16 PASSED [0.0187s] [ 71%] 2024-08-08T21:30:12.0735351Z test_decomp.py::TestDecompCUDA::test_quick_clone_cuda_complex32 PASSED [0.0816s] [ 71%] 2024-08-08T21:30:12.0736868Z test_decomp.py::TestDecompCUDA::test_quick_clone_cuda_complex64 PASSED [0.0807s] [ 71%] 2024-08-08T21:30:12.0738374Z test_decomp.py::TestDecompCUDA::test_quick_clone_cuda_float32 PASSED [0.0503s] [ 72%] 2024-08-08T21:30:12.0739852Z test_decomp.py::TestDecompCUDA::test_quick_complex_cuda_float16 PASSED [0.1244s] [ 72%] 2024-08-08T21:30:12.0741412Z test_decomp.py::TestDecompCUDA::test_quick_conj_physical_cuda_bfloat16 PASSED [0.0053s] [ 72%] 2024-08-08T21:30:12.0743530Z test_decomp.py::TestDecompCUDA::test_quick_conj_physical_cuda_int64 SKIPPED [0.0051s] (only backwards is decomposed, but dtype doesn't support AD) [ 72%] 2024-08-08T21:30:12.0746012Z test_decomp.py::TestDecompCUDA::test_quick_conj_physical_cuda_int8 SKIPPED [0.0048s] (only backwards is decomposed, but dtype doesn't support AD) [ 72%] 2024-08-08T21:30:12.0747973Z test_decomp.py::TestDecompCUDA::test_quick_constant_pad_nd_cuda_int16 PASSED [0.4373s] [ 72%] 2024-08-08T21:30:12.0749715Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_deg2rad_cuda_float64 SKIPPED [0.0003s] (Skipped!) [ 72%] 2024-08-08T21:30:12.0751536Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_expand_copy_cuda_float64 PASSED [90.8160s] [ 72%] 2024-08-08T21:30:12.0753373Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_logaddexp_cuda_float64 SKIPPED [0.0003s] (Skipped!) [ 72%] 2024-08-08T21:30:12.0755304Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_mvlgamma_mvlgamma_p_3_cuda_float64 PASSED [30.9782s] [ 72%] 2024-08-08T21:30:12.0757101Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_nansum_cuda_float64 PASSED [20.4330s] [ 72%] 2024-08-08T21:30:12.0759299Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_norm_cuda_float64 XFAIL [1.4411s] [ 72%] 2024-08-08T21:30:12.0760938Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_norm_inf_cuda_float64 XFAIL [1.4768s] [ 73%] 2024-08-08T21:30:12.0762508Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_norm_nuc_cuda_float64 XFAIL [1.7239s] [ 73%] 2024-08-08T21:30:12.0763835Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_split_with_sizes_copy_cuda_float64 2024-08-08T21:30:12.0764532Z 2024-08-08T21:30:12.0765528Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_decomp/test_decomp-86ac3fdb3ab1fe1f.xml - 2024-08-08T21:30:12.0766797Z !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-08-08T21:30:12.0768146Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_tensor.py:1060: KeyboardInterrupt 2024-08-08T21:30:12.0769544Z (to show a full traceback on KeyboardInterrupt use --full-trace) 2024-08-08T21:30:12.0770886Z =========== 847 passed, 44 skipped, 7 xfailed in 1795.10s (0:29:55) ============ 2024-08-08T21:30:12.0771804Z Got exit code 2 2024-08-08T21:30:12.0772184Z Retrying single test... 2024-08-08T21:30:12.0773185Z Test results will be stored in test-reports/python-pytest/test_decomp/test_decomp-8c4eac52d80c4fc3.xml 2024-08-08T21:30:12.0774365Z ============================= test session starts ============================== 2024-08-08T21:30:12.0775527Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-08T21:30:12.0776422Z cachedir: .pytest_cache 2024-08-08T21:30:12.0777557Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-08T21:30:12.0778654Z rootdir: /var/lib/jenkins/workspace 2024-08-08T21:30:12.0779158Z configfile: pytest.ini 2024-08-08T21:30:12.0780151Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-08T21:30:12.0781398Z collecting ... collected 8751 items / 1227 deselected / 7524 selected 2024-08-08T21:30:12.0782887Z stepcurrent: skipping 898 already run items. Running only test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_split_with_sizes_copy_cuda_float64 2024-08-08T21:30:12.0784149Z Running 1 items in this shard 2024-08-08T21:30:12.0784468Z 2024-08-08T21:30:12.0785361Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_split_with_sizes_copy_cuda_float64 PASSED [199.1900s] [100%] 2024-08-08T21:30:12.0786376Z 2024-08-08T21:30:12.0787412Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_decomp/test_decomp-8c4eac52d80c4fc3.xml - 2024-08-08T21:30:12.0789120Z ================ 1 passed, 1227 deselected in 200.04s (0:03:20) ================ 2024-08-08T21:30:12.0789995Z Got exit code 0 2024-08-08T21:30:12.0790579Z Test succeeeded in new process, continuing with the rest of the tests 2024-08-08T21:30:12.0791870Z Test results will be stored in test-reports/python-pytest/test_decomp/test_decomp-32ed1af9f4a2f5b1.xml 2024-08-08T21:30:12.0793075Z ============================= test session starts ============================== 2024-08-08T21:30:12.0794217Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-08T21:30:12.0795107Z cachedir: .pytest_cache 2024-08-08T21:30:12.0796258Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-08T21:30:12.0797355Z rootdir: /var/lib/jenkins/workspace 2024-08-08T21:30:12.0797839Z configfile: pytest.ini 2024-08-08T21:30:12.0798831Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-08T21:30:12.0800102Z collecting ... collected 8751 items / 899 deselected / 7852 selected 2024-08-08T21:30:12.0800869Z stepcurrent: skipping 899 already run items. 2024-08-08T21:30:12.0801444Z Running 329 items in this shard 2024-08-08T21:30:12.0801768Z 2024-08-08T21:30:12.0802633Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_std_mean_unbiased_cuda_float64 PASSED [3.7858s] [ 0%] 2024-08-08T21:30:12.0804304Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_std_unbiased_cuda_float64 PASSED [1.8552s] [ 0%] 2024-08-08T21:30:12.0805907Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_sum_cuda_float64 PASSED [12.7860s] [ 0%] 2024-08-08T21:30:12.0807471Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_trace_cuda_float64 PASSED [1.2924s] [ 1%] 2024-08-08T21:30:12.0809176Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_tril_cuda_float64 SKIPPED [0.0004s] (Skipped!) [ 1%] 2024-08-08T21:30:12.0810849Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_vdot_cuda_float64 PASSED [0.6885s] [ 1%] 2024-08-08T21:30:12.0812405Z test_decomp.py::TestDecompCUDA::test_quick_core_backward_zero__cuda_float64 XFAIL [0.1248s] [ 2%] 2024-08-08T21:30:12.0813880Z test_decomp.py::TestDecompCUDA::test_quick_cos_cuda_complex128 PASSED [0.4267s] [ 2%] 2024-08-08T21:30:12.0815298Z test_decomp.py::TestDecompCUDA::test_quick_cos_cuda_complex64 PASSED [0.3992s] [ 2%] 2024-08-08T21:30:12.0816678Z test_decomp.py::TestDecompCUDA::test_quick_cosh_cuda_bfloat16 PASSED [0.0114s] [ 3%] 2024-08-08T21:30:12.0818147Z test_decomp.py::TestDecompCUDA::test_quick_count_nonzero_cuda_bfloat16 PASSED [0.0800s] [ 3%] 2024-08-08T21:30:12.0819681Z test_decomp.py::TestDecompCUDA::test_quick_count_nonzero_cuda_complex128 PASSED [0.1727s] [ 3%] 2024-08-08T21:30:12.0821175Z test_decomp.py::TestDecompCUDA::test_quick_count_nonzero_cuda_float64 PASSED [0.1115s] [ 3%] 2024-08-08T21:30:12.0822651Z test_decomp.py::TestDecompCUDA::test_quick_count_nonzero_cuda_int64 PASSED [0.0755s] [ 4%] 2024-08-08T21:30:12.0824111Z test_decomp.py::TestDecompCUDA::test_quick_cumprod_cuda_complex64 PASSED [2.2051s] [ 4%] 2024-08-08T21:30:12.0825534Z test_decomp.py::TestDecompCUDA::test_quick_cumprod_cuda_int32 PASSED [0.1302s] [ 4%] 2024-08-08T21:30:12.0826945Z test_decomp.py::TestDecompCUDA::test_quick_cumsum_cuda_complex128 PASSED [0.1228s] [ 5%] 2024-08-08T21:30:12.0828432Z test_decomp.py::TestDecompCUDA::test_quick_deg2rad_cuda_uint8 PASSED [0.0408s] [ 5%] 2024-08-08T21:30:12.0830157Z test_decomp.py::TestDecompCUDA::test_quick_diag_cuda_float16 SKIPPED [0.0027s] (diag in torch.float16 not supported) [ 5%] 2024-08-08T21:30:12.0832000Z test_decomp.py::TestDecompCUDA::test_quick_diag_cuda_int32 SKIPPED [0.0028s] (diag in torch.int32 not supported) [ 6%] 2024-08-08T21:30:12.0833835Z test_decomp.py::TestDecompCUDA::test_quick_diag_cuda_int64 SKIPPED [0.0027s] (diag in torch.int64 not supported) [ 6%] 2024-08-08T21:30:12.0835438Z test_decomp.py::TestDecompCUDA::test_quick_diag_embed_cuda_bool PASSED [0.1272s] [ 6%] 2024-08-08T21:30:12.0836905Z test_decomp.py::TestDecompCUDA::test_quick_diag_embed_cuda_complex128 PASSED [0.2518s] [ 6%] 2024-08-08T21:30:12.0838364Z test_decomp.py::TestDecompCUDA::test_quick_diag_embed_cuda_float64 PASSED [0.1604s] [ 7%] 2024-08-08T21:30:12.0839815Z test_decomp.py::TestDecompCUDA::test_quick_diag_embed_cuda_uint8 PASSED [0.1076s] [ 7%] 2024-08-08T21:30:12.0841304Z test_decomp.py::TestDecompCUDA::test_quick_diagonal_copy_cuda_bfloat16 PASSED [0.0294s] [ 7%] 2024-08-08T21:30:12.0842819Z test_decomp.py::TestDecompCUDA::test_quick_diagonal_copy_cuda_complex32 PASSED [0.2572s] [ 8%] 2024-08-08T21:30:12.0844290Z test_decomp.py::TestDecompCUDA::test_quick_diagonal_copy_cuda_int16 PASSED [0.1014s] [ 8%] 2024-08-08T21:30:12.0845756Z test_decomp.py::TestDecompCUDA::test_quick_diagonal_copy_cuda_int8 PASSED [0.1015s] [ 8%] 2024-08-08T21:30:12.0847217Z test_decomp.py::TestDecompCUDA::test_quick_diagonal_cuda_complex64 PASSED [0.3676s] [ 9%] 2024-08-08T21:30:12.0848637Z test_decomp.py::TestDecompCUDA::test_quick_diagonal_cuda_uint8 PASSED [0.0988s] [ 9%] 2024-08-08T21:30:12.0850129Z test_decomp.py::TestDecompCUDA::test_quick_diagonal_scatter_cuda_complex64 PASSED [1.0308s] [ 9%] 2024-08-08T21:30:12.0851738Z test_decomp.py::TestDecompCUDA::test_quick_diagonal_scatter_cuda_float32 PASSED [0.5795s] [ 10%] 2024-08-08T21:30:12.0853240Z test_decomp.py::TestDecompCUDA::test_quick_digamma_cuda_bool PASSED [0.0160s] [ 10%] 2024-08-08T21:30:12.0854673Z test_decomp.py::TestDecompCUDA::test_quick_digamma_cuda_int64 PASSED [0.0127s] [ 10%] 2024-08-08T21:30:12.0856140Z test_decomp.py::TestDecompCUDA::test_quick_div_floor_rounding_cuda_int64 PASSED [0.1910s] [ 10%] 2024-08-08T21:30:12.0857714Z test_decomp.py::TestDecompCUDA::test_quick_div_no_rounding_mode_cuda_complex64 PASSED [0.3572s] [ 11%] 2024-08-08T21:30:12.0859522Z test_decomp.py::TestDecompCUDA::test_quick_div_no_rounding_mode_cuda_float32 PASSED [0.2133s] [ 11%] 2024-08-08T21:30:12.0861077Z test_decomp.py::TestDecompCUDA::test_quick_div_no_rounding_mode_cuda_int32 PASSED [0.1552s] [ 11%] 2024-08-08T21:30:12.0862628Z test_decomp.py::TestDecompCUDA::test_quick_div_no_rounding_mode_cuda_uint8 PASSED [0.1537s] [ 12%] 2024-08-08T21:30:12.0864446Z test_decomp.py::TestDecompCUDA::test_quick_empty_like_cuda_float16 SKIPPED [0.0027s] (empty_like in torch.float16 not supported) [ 12%] 2024-08-08T21:30:12.0866456Z test_decomp.py::TestDecompCUDA::test_quick_empty_like_cuda_float32 SKIPPED [0.0028s] (empty_like in torch.float32 not supported) [ 12%] 2024-08-08T21:30:12.0868439Z test_decomp.py::TestDecompCUDA::test_quick_empty_like_cuda_int16 SKIPPED [0.0030s] (empty_like in torch.int16 not supported) [ 13%] 2024-08-08T21:30:12.0870399Z test_decomp.py::TestDecompCUDA::test_quick_empty_like_cuda_int32 SKIPPED [0.0027s] (empty_like in torch.int32 not supported) [ 13%] 2024-08-08T21:30:12.0872021Z test_decomp.py::TestDecompCUDA::test_quick_eq_cuda_int32 PASSED [0.1569s] [ 13%] 2024-08-08T21:30:12.0873451Z test_decomp.py::TestDecompCUDA::test_quick_erf_cuda_bfloat16 PASSED [0.0123s] [ 13%] 2024-08-08T21:30:12.0874904Z test_decomp.py::TestDecompCUDA::test_quick_erf_cuda_float32 PASSED [0.0515s] [ 14%] 2024-08-08T21:30:12.0876260Z test_decomp.py::TestDecompCUDA::test_quick_erf_cuda_uint8 PASSED [0.0395s] [ 14%] 2024-08-08T21:30:12.0877601Z test_decomp.py::TestDecompCUDA::test_quick_erfc_cuda_bool PASSED [0.2034s] [ 14%] 2024-08-08T21:30:12.0878975Z test_decomp.py::TestDecompCUDA::test_quick_erfinv_cuda_bfloat16 PASSED [0.2735s] [ 15%] 2024-08-08T21:30:12.0880365Z test_decomp.py::TestDecompCUDA::test_quick_erfinv_cuda_bool PASSED [0.0438s] [ 15%] 2024-08-08T21:30:12.0881738Z test_decomp.py::TestDecompCUDA::test_quick_erfinv_cuda_int16 PASSED [0.0415s] [ 15%] 2024-08-08T21:30:12.0883116Z test_decomp.py::TestDecompCUDA::test_quick_exp_cuda_complex64 PASSED [0.3664s] [ 16%] 2024-08-08T21:30:12.0884495Z test_decomp.py::TestDecompCUDA::test_quick_exp_cuda_uint8 PASSED [0.0136s] [ 16%] 2024-08-08T21:30:12.0885898Z test_decomp.py::TestDecompCUDA::test_quick_expand_copy_cuda_bool PASSED [0.0348s] [ 16%] 2024-08-08T21:30:12.0887341Z test_decomp.py::TestDecompCUDA::test_quick_expand_copy_cuda_int64 PASSED [0.0356s] [ 17%] 2024-08-08T21:30:12.0888781Z test_decomp.py::TestDecompCUDA::test_quick_expand_cuda_complex128 PASSED [0.0931s] [ 17%] 2024-08-08T21:30:12.0890193Z test_decomp.py::TestDecompCUDA::test_quick_expand_cuda_uint8 PASSED [0.0338s] [ 17%] 2024-08-08T21:30:12.0891588Z test_decomp.py::TestDecompCUDA::test_quick_expm1_cuda_float16 PASSED [0.0079s] [ 17%] 2024-08-08T21:30:12.0893006Z test_decomp.py::TestDecompCUDA::test_quick_exponential_cuda_float16 XFAIL [0.0126s] [ 18%] 2024-08-08T21:30:12.0894415Z test_decomp.py::TestDecompCUDA::test_quick_eye_cuda_uint8 PASSED [0.1371s] [ 18%] 2024-08-08T21:30:12.0895896Z test_decomp.py::TestDecompCUDA::test_quick_fft_fft2_cuda_complex64 PASSED [0.1562s] [ 18%] 2024-08-08T21:30:12.0897831Z test_decomp.py::TestDecompCUDA::test_quick_fft_fft2_cuda_int64 SKIPPED [0.0053s] (only backwards is decomposed, but dtype doesn't support AD) [ 19%] 2024-08-08T21:30:12.0900009Z test_decomp.py::TestDecompCUDA::test_quick_fft_fft2_cuda_int8 SKIPPED [0.0050s] (only backwards is decomposed, but dtype doesn't support AD) [ 19%] 2024-08-08T21:30:12.0901808Z test_decomp.py::TestDecompCUDA::test_quick_fft_fftn_cuda_complex128 PASSED [0.1565s] [ 19%] 2024-08-08T21:30:12.0903270Z test_decomp.py::TestDecompCUDA::test_quick_fft_hfft_cuda_complex64 PASSED [0.3978s] [ 20%] 2024-08-08T21:30:12.0904727Z test_decomp.py::TestDecompCUDA::test_quick_fft_ifft2_cuda_float32 PASSED [0.2831s] [ 20%] 2024-08-08T21:30:12.0906577Z test_decomp.py::TestDecompCUDA::test_quick_fft_ifft2_cuda_int32 SKIPPED [0.0052s] (only backwards is decomposed, but dtype doesn't support AD) [ 20%] 2024-08-08T21:30:12.0908386Z test_decomp.py::TestDecompCUDA::test_quick_fft_ifft_cuda_float32 PASSED [0.0509s] [ 20%] 2024-08-08T21:30:12.0910228Z test_decomp.py::TestDecompCUDA::test_quick_fft_ifft_cuda_int32 SKIPPED [0.0050s] (only backwards is decomposed, but dtype doesn't support AD) [ 21%] 2024-08-08T21:30:12.0912426Z test_decomp.py::TestDecompCUDA::test_quick_fft_ifft_cuda_int64 SKIPPED [0.0050s] (only backwards is decomposed, but dtype doesn't support AD) [ 21%] 2024-08-08T21:30:12.0914599Z test_decomp.py::TestDecompCUDA::test_quick_fft_ifft_cuda_int8 SKIPPED [0.0052s] (only backwards is decomposed, but dtype doesn't support AD) [ 21%] 2024-08-08T21:30:12.0916857Z test_decomp.py::TestDecompCUDA::test_quick_fft_ifftn_cuda_int32 SKIPPED [0.0049s] (only backwards is decomposed, but dtype doesn't support AD) [ 22%] 2024-08-08T21:30:12.0918721Z test_decomp.py::TestDecompCUDA::test_quick_fft_ihfft2_cuda_float32 PASSED [0.0983s] [ 22%] 2024-08-08T21:30:12.0920589Z test_decomp.py::TestDecompCUDA::test_quick_fft_ihfft_cuda_int64 SKIPPED [0.0050s] (only backwards is decomposed, but dtype doesn't support AD) [ 22%] 2024-08-08T21:30:12.0922777Z test_decomp.py::TestDecompCUDA::test_quick_fft_irfft_cuda_int64 SKIPPED [0.0048s] (only backwards is decomposed, but dtype doesn't support AD) [ 23%] 2024-08-08T21:30:12.0924601Z test_decomp.py::TestDecompCUDA::test_quick_fft_irfftn_cuda_complex128 PASSED [0.1602s] [ 23%] 2024-08-08T21:30:12.0926489Z test_decomp.py::TestDecompCUDA::test_quick_fft_irfftn_cuda_int32 SKIPPED [0.0052s] (only backwards is decomposed, but dtype doesn't support AD) [ 23%] 2024-08-08T21:30:12.0928690Z test_decomp.py::TestDecompCUDA::test_quick_fft_rfft_cuda_bool SKIPPED [0.0054s] (only backwards is decomposed, but dtype doesn't support AD) [ 24%] 2024-08-08T21:30:12.0930450Z test_decomp.py::TestDecompCUDA::test_quick_fill_cuda_complex128 PASSED [0.0242s] [ 24%] 2024-08-08T21:30:12.0931130Z test_decomp.py::TestDecompCUDA::test_quick_fill_cuda_float16 PASSED [0.0089s] [ 24%] 2024-08-08T21:30:12.0931770Z test_decomp.py::TestDecompCUDA::test_quick_fill_cuda_int16 PASSED [0.0114s] [ 24%] 2024-08-08T21:30:12.0932414Z test_decomp.py::TestDecompCUDA::test_quick_fill_cuda_int32 PASSED [0.0111s] [ 25%] 2024-08-08T21:30:12.0933067Z test_decomp.py::TestDecompCUDA::test_quick_flip_cuda_float16 PASSED [0.0203s] [ 25%] 2024-08-08T21:30:12.0933726Z test_decomp.py::TestDecompCUDA::test_quick_floor_cuda_float16 PASSED [0.0070s] [ 25%] 2024-08-08T21:30:12.0934460Z test_decomp.py::TestDecompCUDA::test_quick_floor_divide_cuda_bfloat16 PASSED [0.0208s] [ 26%] 2024-08-08T21:30:12.0935109Z test_decomp.py::TestDecompCUDA::test_quick_fmin_cuda_float64 PASSED [0.2285s] [ 26%] 2024-08-08T21:30:12.0935800Z test_decomp.py::TestDecompCUDA::test_quick_fmin_cuda_int32 PASSED [0.1497s] [ 26%] 2024-08-08T21:30:12.0936524Z test_decomp.py::TestDecompCUDA::test_quick_fmod_cuda_bfloat16 PASSED [0.0251s] [ 27%] 2024-08-08T21:30:12.0937165Z test_decomp.py::TestDecompCUDA::test_quick_fmod_cuda_uint8 PASSED [0.1497s] [ 27%] 2024-08-08T21:30:12.0937829Z test_decomp.py::TestDecompCUDA::test_quick_full_cuda_bfloat16 PASSED [0.0107s] [ 27%] 2024-08-08T21:30:12.0938479Z test_decomp.py::TestDecompCUDA::test_quick_full_cuda_float64 PASSED [0.0116s] [ 27%] 2024-08-08T21:30:12.0939115Z test_decomp.py::TestDecompCUDA::test_quick_full_cuda_uint8 PASSED [0.0101s] [ 28%] 2024-08-08T21:30:12.0939755Z test_decomp.py::TestDecompCUDA::test_quick_gcd_cuda_int16 PASSED [0.6802s] [ 28%] 2024-08-08T21:30:12.0940384Z test_decomp.py::TestDecompCUDA::test_quick_gcd_cuda_int32 PASSED [0.1504s] [ 28%] 2024-08-08T21:30:12.0941021Z test_decomp.py::TestDecompCUDA::test_quick_ge_cuda_float32 PASSED [0.2090s] [ 29%] 2024-08-08T21:30:12.0941699Z test_decomp.py::TestDecompCUDA::test_quick_geometric_cuda_int8 XFAIL [0.0088s] [ 29%] 2024-08-08T21:30:12.0942372Z test_decomp.py::TestDecompCUDA::test_quick_heaviside_cuda_int16 PASSED [0.1624s] [ 29%] 2024-08-08T21:30:12.0943008Z test_decomp.py::TestDecompCUDA::test_quick_i0_cuda_uint8 PASSED [0.0133s] [ 30%] 2024-08-08T21:30:12.0943677Z test_decomp.py::TestDecompCUDA::test_quick_igammac_cuda_float64 PASSED [0.2141s] [ 30%] 2024-08-08T21:30:12.0944357Z test_decomp.py::TestDecompCUDA::test_quick_index_add_cuda_float64 PASSED [0.1208s] [ 30%] 2024-08-08T21:30:12.0945088Z test_decomp.py::TestDecompCUDA::test_quick_index_add_cuda_uint8 PASSED [0.0558s] [ 31%] 2024-08-08T21:30:12.0945849Z test_decomp.py::TestDecompCUDA::test_quick_index_copy_cuda_complex128 PASSED [0.0543s] [ 31%] 2024-08-08T21:30:12.0946570Z test_decomp.py::TestDecompCUDA::test_quick_index_fill_cuda_complex128 PASSED [0.0777s] [ 31%] 2024-08-08T21:30:12.0947284Z test_decomp.py::TestDecompCUDA::test_quick_index_select_cuda_complex32 PASSED [0.0348s] [ 31%] 2024-08-08T21:30:12.0947969Z test_decomp.py::TestDecompCUDA::test_quick_index_select_cuda_int16 PASSED [0.0159s] [ 32%] 2024-08-08T21:30:12.0948607Z test_decomp.py::TestDecompCUDA::test_quick_isin_cuda_int32 PASSED [0.0551s] [ 32%] 2024-08-08T21:30:12.0949260Z test_decomp.py::TestDecompCUDA::test_quick_isnan_cuda_float64 PASSED [0.0514s] [ 32%] 2024-08-08T21:30:12.0949908Z test_decomp.py::TestDecompCUDA::test_quick_isnan_cuda_int16 PASSED [0.0397s] [ 33%] 2024-08-08T21:30:12.0950557Z test_decomp.py::TestDecompCUDA::test_quick_isnan_cuda_int8 PASSED [0.0390s] [ 33%] 2024-08-08T21:30:12.0951210Z test_decomp.py::TestDecompCUDA::test_quick_isneginf_cuda_bool PASSED [0.0397s] [ 33%] 2024-08-08T21:30:12.0952113Z test_decomp.py::TestDecompCUDA::test_quick_item_cuda_int32 SKIPPED [0.0027s] (item in torch.int32 not supported) [ 34%] 2024-08-08T21:30:12.0952734Z test_decomp.py::TestDecompCUDA::test_quick_lcm_cuda_int64 PASSED [1.3837s] [ 34%] 2024-08-08T21:30:12.0953387Z test_decomp.py::TestDecompCUDA::test_quick_lerp_cuda_bfloat16 PASSED [0.0416s] [ 34%] 2024-08-08T21:30:12.0954083Z test_decomp.py::TestDecompCUDA::test_quick_lgamma_cuda_bool PASSED [0.2750s] [ 34%] 2024-08-08T21:30:12.0954734Z test_decomp.py::TestDecompCUDA::test_quick_lgamma_cuda_int32 PASSED [0.0131s] [ 35%] 2024-08-08T21:30:12.0955398Z test_decomp.py::TestDecompCUDA::test_quick_lgamma_cuda_int64 PASSED [0.0128s] [ 35%] 2024-08-08T21:30:12.0956142Z test_decomp.py::TestDecompCUDA::test_quick_linalg_cross_cuda_uint8 PASSED [0.0396s] [ 35%] 2024-08-08T21:30:12.0957359Z test_decomp.py::TestDecompCUDA::test_quick_linalg_diagonal_cuda_int32 SKIPPED [0.0049s] (only backwards is decomposed, but dtype doesn't support AD) [ 36%] 2024-08-08T21:30:12.0958055Z test_decomp.py::TestDecompCUDA::test_quick_linspace_cuda_float16 PASSED [0.0899s] [ 36%] 2024-08-08T21:30:12.0958921Z test_decomp.py::TestDecompCUDA::test_quick_linspace_cuda_float64 PASSED [0.1079s] [ 36%] 2024-08-08T21:30:12.0959612Z test_decomp.py::TestDecompCUDA::test_quick_linspace_cuda_int32 PASSED [0.1054s] [ 37%] 2024-08-08T21:30:12.0960422Z test_decomp.py::TestDecompCUDA::test_quick_linspace_tensor_overload_cuda_float32 PASSED [0.5984s] [ 37%] 2024-08-08T21:30:12.0961069Z test_decomp.py::TestDecompCUDA::test_quick_log10_cuda_bool PASSED [0.0189s] [ 37%] 2024-08-08T21:30:12.0961783Z test_decomp.py::TestDecompCUDA::test_quick_log10_cuda_complex128 PASSED [0.2741s] [ 37%] 2024-08-08T21:30:12.0962444Z test_decomp.py::TestDecompCUDA::test_quick_log2_cuda_float64 PASSED [0.0176s] [ 38%] 2024-08-08T21:30:12.0963104Z test_decomp.py::TestDecompCUDA::test_quick_log_cuda_complex32 PASSED [0.3906s] [ 38%] 2024-08-08T21:30:12.0963784Z test_decomp.py::TestDecompCUDA::test_quick_log_cuda_complex64 PASSED [0.0248s] [ 38%] 2024-08-08T21:30:12.0964491Z test_decomp.py::TestDecompCUDA::test_quick_log_softmax_cuda_bfloat16 PASSED [0.0400s] [ 39%] 2024-08-08T21:30:12.0965206Z test_decomp.py::TestDecompCUDA::test_quick_log_softmax_cuda_float16 PASSED [0.0275s] [ 39%] 2024-08-08T21:30:12.0965983Z test_decomp.py::TestDecompCUDA::test_quick_log_softmax_cuda_float32 PASSED [0.2110s] [ 39%] 2024-08-08T21:30:12.0966750Z test_decomp.py::TestDecompCUDA::test_quick_logaddexp2_cuda_float32 PASSED [0.0236s] [ 40%] 2024-08-08T21:30:12.0967460Z test_decomp.py::TestDecompCUDA::test_quick_logaddexp_cuda_float32 PASSED [0.2154s] [ 40%] 2024-08-08T21:30:12.0968166Z test_decomp.py::TestDecompCUDA::test_quick_logical_and_cuda_float32 PASSED [0.2066s] [ 40%] 2024-08-08T21:30:12.0968846Z test_decomp.py::TestDecompCUDA::test_quick_logical_and_cuda_int64 PASSED [0.1483s] [ 41%] 2024-08-08T21:30:12.0969534Z test_decomp.py::TestDecompCUDA::test_quick_logical_and_cuda_int8 PASSED [0.1474s] [ 41%] 2024-08-08T21:30:12.0970216Z test_decomp.py::TestDecompCUDA::test_quick_logical_not_cuda_int8 PASSED [0.0114s] [ 41%] 2024-08-08T21:30:12.0970926Z test_decomp.py::TestDecompCUDA::test_quick_logical_or_cuda_float16 PASSED [0.1220s] [ 41%] 2024-08-08T21:30:12.0971603Z test_decomp.py::TestDecompCUDA::test_quick_logical_or_cuda_int64 PASSED [0.1510s] [ 42%] 2024-08-08T21:30:12.0972273Z test_decomp.py::TestDecompCUDA::test_quick_logit_cuda_float64 PASSED [0.1595s] [ 42%] 2024-08-08T21:30:12.0972947Z test_decomp.py::TestDecompCUDA::test_quick_logit_cuda_int64 PASSED [0.0430s] [ 42%] 2024-08-08T21:30:12.0973614Z test_decomp.py::TestDecompCUDA::test_quick_logspace_cuda_int16 XFAIL [0.1040s] [ 43%] 2024-08-08T21:30:12.0974295Z test_decomp.py::TestDecompCUDA::test_quick_logspace_cuda_int64 XFAIL [0.0753s] [ 43%] 2024-08-08T21:30:12.0974989Z test_decomp.py::TestDecompCUDA::test_quick_logsumexp_cuda_bfloat16 PASSED [0.0190s] [ 43%] 2024-08-08T21:30:12.0975676Z test_decomp.py::TestDecompCUDA::test_quick_logsumexp_cuda_float16 PASSED [0.0186s] [ 44%] 2024-08-08T21:30:12.0976378Z test_decomp.py::TestDecompCUDA::test_quick_logsumexp_cuda_float64 PASSED [0.1202s] [ 44%] 2024-08-08T21:30:12.0977057Z test_decomp.py::TestDecompCUDA::test_quick_logsumexp_cuda_int16 PASSED [0.0312s] [ 44%] 2024-08-08T21:30:12.0977770Z test_decomp.py::TestDecompCUDA::test_quick_lt_cuda_bfloat16 PASSED [0.1179s] [ 44%] 2024-08-08T21:30:12.0978496Z test_decomp.py::TestDecompCUDA::test_quick_lt_cuda_float32 PASSED [0.2094s] [ 45%] 2024-08-08T21:30:12.0979181Z test_decomp.py::TestDecompCUDA::test_quick_masked_fill_cuda_int16 PASSED [0.0598s] [ 45%] 2024-08-08T21:30:12.0979860Z test_decomp.py::TestDecompCUDA::test_quick_maximum_cuda_int16 PASSED [0.1508s] [ 45%] 2024-08-08T21:30:12.0980915Z test_decomp.py::TestDecompCUDA::test_quick_meshgrid_list_of_tensors_cuda_int16 SKIPPED [0.0028s] (meshgrid in torch.int16 not supported) [ 46%] 2024-08-08T21:30:12.0981985Z test_decomp.py::TestDecompCUDA::test_quick_meshgrid_variadic_tensors_cuda_int16 SKIPPED [0.0027s] (meshgrid in torch.int16 not supported) [ 46%] 2024-08-08T21:30:12.0982676Z test_decomp.py::TestDecompCUDA::test_quick_minimum_cuda_bool PASSED [0.1500s] [ 46%] 2024-08-08T21:30:12.0983356Z test_decomp.py::TestDecompCUDA::test_quick_minimum_cuda_float64 PASSED [0.2142s] [ 47%] 2024-08-08T21:30:12.0984036Z test_decomp.py::TestDecompCUDA::test_quick_mul_cuda_complex64 PASSED [0.3599s] [ 47%] 2024-08-08T21:30:12.0984680Z test_decomp.py::TestDecompCUDA::test_quick_mul_cuda_float32 PASSED [0.2136s] [ 47%] 2024-08-08T21:30:12.0985310Z test_decomp.py::TestDecompCUDA::test_quick_mul_cuda_int64 PASSED [0.1507s] [ 48%] 2024-08-08T21:30:12.0985970Z test_decomp.py::TestDecompCUDA::test_quick_mv_cuda_bfloat16 PASSED [0.0236s] [ 48%] 2024-08-08T21:30:12.0986738Z test_decomp.py::TestDecompCUDA::test_quick_mvlgamma_mvlgamma_p_3_cuda_float32 PASSED [0.0702s] [ 48%] 2024-08-08T21:30:12.0987538Z test_decomp.py::TestDecompCUDA::test_quick_mvlgamma_mvlgamma_p_3_cuda_int64 PASSED [0.0454s] [ 48%] 2024-08-08T21:30:12.0988358Z test_decomp.py::TestDecompCUDA::test_quick_mvlgamma_mvlgamma_p_3_cuda_uint8 PASSED [0.0453s] [ 49%] 2024-08-08T21:30:12.0989058Z test_decomp.py::TestDecompCUDA::test_quick_nan_to_num_cuda_float16 PASSED [0.0109s] [ 49%] 2024-08-08T21:30:12.0989755Z test_decomp.py::TestDecompCUDA::test_quick_nan_to_num_cuda_int16 PASSED [0.0110s] [ 49%] 2024-08-08T21:30:12.0990425Z test_decomp.py::TestDecompCUDA::test_quick_nan_to_num_cuda_uint8 PASSED [0.0115s] [ 50%] 2024-08-08T21:30:12.0991068Z test_decomp.py::TestDecompCUDA::test_quick_nansum_cuda_bool PASSED [0.0843s] [ 50%] 2024-08-08T21:30:12.0991795Z test_decomp.py::TestDecompCUDA::test_quick_narrow_copy_cuda_complex64 PASSED [0.1864s] [ 50%] 2024-08-08T21:30:12.0992487Z test_decomp.py::TestDecompCUDA::test_quick_narrow_copy_cuda_int32 PASSED [0.0849s] [ 51%] 2024-08-08T21:30:12.0993612Z test_decomp.py::TestDecompCUDA::test_quick_native_batch_norm_cuda_float16 SKIPPED [0.0028s] (native_batch_norm in torch.float16 not supported) [ 51%] 2024-08-08T21:30:12.0994359Z test_decomp.py::TestDecompCUDA::test_quick_native_layer_norm_cuda_float32 PASSED [0.4298s] [ 51%] 2024-08-08T21:30:12.0995001Z test_decomp.py::TestDecompCUDA::test_quick_ne_cuda_float32 PASSED [0.2117s] [ 51%] 2024-08-08T21:30:12.0995645Z test_decomp.py::TestDecompCUDA::test_quick_ne_cuda_uint8 PASSED [0.1496s] [ 52%] 2024-08-08T21:30:12.0996290Z test_decomp.py::TestDecompCUDA::test_quick_neg_cuda_float32 PASSED [0.0534s] [ 52%] 2024-08-08T21:30:12.0996938Z test_decomp.py::TestDecompCUDA::test_quick_neg_cuda_int64 PASSED [0.0412s] [ 52%] 2024-08-08T21:30:12.0997884Z test_decomp.py::TestDecompCUDA::test_quick_new_empty_cuda_bool SKIPPED [0.0028s] (new_empty in torch.bool not supported) [ 53%] 2024-08-08T21:30:12.0998885Z test_decomp.py::TestDecompCUDA::test_quick_new_empty_cuda_int8 SKIPPED [0.0027s] (new_empty in torch.int8 not supported) [ 53%] 2024-08-08T21:30:12.1000008Z test_decomp.py::TestDecompCUDA::test_quick_new_empty_strided_cuda_int64 SKIPPED [0.0003s] (Expected: new_empty_strided is not comparable) [ 53%] 2024-08-08T21:30:12.1000715Z test_decomp.py::TestDecompCUDA::test_quick_new_ones_cuda_complex128 PASSED [0.0643s] [ 54%] 2024-08-08T21:30:12.1001416Z test_decomp.py::TestDecompCUDA::test_quick_new_ones_cuda_float16 PASSED [0.0162s] [ 54%] 2024-08-08T21:30:12.1002111Z test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_bfloat16 PASSED [0.0161s] [ 54%] 2024-08-08T21:30:12.1002781Z test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_bool PASSED [0.0249s] [ 55%] 2024-08-08T21:30:12.1003509Z test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_complex128 PASSED [0.0633s] [ 55%] 2024-08-08T21:30:12.1004201Z test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_float64 PASSED [0.0414s] [ 55%] 2024-08-08T21:30:12.1004878Z test_decomp.py::TestDecompCUDA::test_quick_new_zeros_cuda_int16 PASSED [0.0246s] [ 55%] 2024-08-08T21:30:12.1005588Z test_decomp.py::TestDecompCUDA::test_quick_nextafter_cuda_float32 PASSED [0.2157s] [ 56%] 2024-08-08T21:30:12.1006391Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_hardsigmoid_cuda_float64 PASSED [0.0394s] [ 56%] 2024-08-08T21:30:12.1007193Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_hardswish_cuda_float32 PASSED [0.1952s] [ 56%] 2024-08-08T21:30:12.1008040Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_leaky_relu_cuda_float64 PASSED [0.4169s] [ 57%] 2024-08-08T21:30:12.1008886Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_logsigmoid_cuda_float16 PASSED [0.0203s] [ 57%] 2024-08-08T21:30:12.1009750Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_pad_constant_cuda_bool PASSED [0.4273s] [ 57%] 2024-08-08T21:30:12.1010563Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_pad_constant_cuda_float64 PASSED [0.5898s] [ 58%] 2024-08-08T21:30:12.1011342Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_prelu_cuda_float32 PASSED [1.9321s] [ 58%] 2024-08-08T21:30:12.1012089Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_relu_cuda_float32 PASSED [0.0630s] [ 58%] 2024-08-08T21:30:12.1013247Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_rrelu_cuda_bfloat16 SKIPPED [0.0029s] (nn.functional.rrelu in torch.bfloat16 not supported) [ 58%] 2024-08-08T21:30:12.1014051Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_softplus_cuda_bfloat16 PASSED [0.0184s] [ 59%] 2024-08-08T21:30:12.1014818Z test_decomp.py::TestDecompCUDA::test_quick_nn_functional_unfold_cuda_float32 PASSED [2.4475s] [ 59%] 2024-08-08T21:30:12.1015828Z test_decomp.py::TestDecompCUDA::test_quick_norm_inf_cuda_complex64 SKIPPED [0.0054s] (norm in torch.complex64 not supported) [ 59%] 2024-08-08T21:30:12.1016778Z test_decomp.py::TestDecompCUDA::test_quick_norm_nuc_cuda_float32 SKIPPED [0.0054s] (norm in torch.float32 not supported) [ 60%] 2024-08-08T21:30:12.1017538Z test_decomp.py::TestDecompCUDA::test_quick_normal_cuda_float16 SKIPPED [0.0004s] (Skipped!) [ 60%] 2024-08-08T21:30:12.1018273Z test_decomp.py::TestDecompCUDA::test_quick_normal_in_place_cuda_float16 XFAIL [0.0122s] [ 60%] 2024-08-08T21:30:12.1019112Z test_decomp.py::TestDecompCUDA::test_quick_normal_number_mean_cuda_float64 SKIPPED [0.0002s] (Skipped!) [ 61%] 2024-08-08T21:30:12.1019817Z test_decomp.py::TestDecompCUDA::test_quick_ones_like_cuda_complex32 PASSED [0.0630s] [ 61%] 2024-08-08T21:30:12.1020590Z test_decomp.py::TestDecompCUDA::test_quick_ones_like_cuda_complex64 PASSED [0.0621s] [ 61%] 2024-08-08T21:30:12.1021322Z test_decomp.py::TestDecompCUDA::test_quick_permute_cuda_bfloat16 PASSED [0.0113s] [ 62%] 2024-08-08T21:30:12.1022002Z test_decomp.py::TestDecompCUDA::test_quick_permute_cuda_int16 PASSED [0.0207s] [ 62%] 2024-08-08T21:30:12.1022661Z test_decomp.py::TestDecompCUDA::test_quick_polar_cuda_float64 PASSED [0.2108s] [ 62%] 2024-08-08T21:30:12.1023317Z test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_bfloat16 PASSED [0.0238s] [ 62%] 2024-08-08T21:30:12.1023994Z test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_complex64 PASSED [0.3471s] [ 63%] 2024-08-08T21:30:12.1024641Z test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_float16 PASSED [0.0235s] [ 63%] 2024-08-08T21:30:12.1025287Z test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_int8 PASSED [0.1429s] [ 63%] 2024-08-08T21:30:12.1025927Z test_decomp.py::TestDecompCUDA::test_quick_pow_cuda_uint8 PASSED [0.1442s] [ 64%] 2024-08-08T21:30:12.1026592Z test_decomp.py::TestDecompCUDA::test_quick_prod_cuda_bfloat16 PASSED [1.0349s] [ 64%] 2024-08-08T21:30:12.1027237Z test_decomp.py::TestDecompCUDA::test_quick_prod_cuda_bool PASSED [0.3847s] [ 64%] 2024-08-08T21:30:12.1027908Z test_decomp.py::TestDecompCUDA::test_quick_prod_cuda_complex128 PASSED [0.8852s] [ 65%] 2024-08-08T21:30:12.1028577Z test_decomp.py::TestDecompCUDA::test_quick_prod_cuda_complex64 PASSED [0.8344s] [ 65%] 2024-08-08T21:30:12.1029281Z test_decomp.py::TestDecompCUDA::test_quick_rad2deg_cuda_bfloat16 PASSED [0.0067s] [ 65%] 2024-08-08T21:30:12.1029938Z test_decomp.py::TestDecompCUDA::test_quick_randn_cuda_bfloat16 XFAIL [0.0071s] [ 65%] 2024-08-08T21:30:12.1030662Z test_decomp.py::TestDecompCUDA::test_quick_randn_cuda_float32 XFAIL [0.0066s] [ 66%] 2024-08-08T21:30:12.1031408Z test_decomp.py::TestDecompCUDA::test_quick_remainder_cuda_bfloat16 PASSED [0.0241s] [ 66%] 2024-08-08T21:30:12.1032105Z test_decomp.py::TestDecompCUDA::test_quick_remainder_cuda_float16 PASSED [0.0232s] [ 66%] 2024-08-08T21:30:12.1032778Z test_decomp.py::TestDecompCUDA::test_quick_repeat_cuda_int16 PASSED [0.0792s] [ 67%] 2024-08-08T21:30:12.1033435Z test_decomp.py::TestDecompCUDA::test_quick_repeat_cuda_int64 PASSED [0.0776s] [ 67%] 2024-08-08T21:30:12.1034096Z test_decomp.py::TestDecompCUDA::test_quick_roll_cuda_float64 PASSED [0.1937s] [ 67%] 2024-08-08T21:30:12.1034734Z test_decomp.py::TestDecompCUDA::test_quick_roll_cuda_int16 PASSED [0.1429s] [ 68%] 2024-08-08T21:30:12.1035403Z test_decomp.py::TestDecompCUDA::test_quick_round_cuda_float32 PASSED [0.0511s] [ 68%] 2024-08-08T21:30:12.1036081Z test_decomp.py::TestDecompCUDA::test_quick_round_cuda_int64 PASSED [0.0383s] [ 68%] 2024-08-08T21:30:12.1036818Z test_decomp.py::TestDecompCUDA::test_quick_round_decimals_3_cuda_float32 PASSED [0.0179s] [ 68%] 2024-08-08T21:30:12.1037588Z test_decomp.py::TestDecompCUDA::test_quick_round_decimals_neg_3_cuda_bfloat16 PASSED [0.0105s] [ 69%] 2024-08-08T21:30:12.1038361Z test_decomp.py::TestDecompCUDA::test_quick_round_decimals_neg_3_cuda_float64 PASSED [0.0174s] [ 69%] 2024-08-08T21:30:12.1039009Z test_decomp.py::TestDecompCUDA::test_quick_rsqrt_cuda_int32 PASSED [0.0129s] [ 69%] 2024-08-08T21:30:12.1039669Z test_decomp.py::TestDecompCUDA::test_quick_rsqrt_cuda_uint8 PASSED [0.0123s] [ 70%] 2024-08-08T21:30:12.1040338Z test_decomp.py::TestDecompCUDA::test_quick_rsub_cuda_complex64 PASSED [0.3897s] [ 70%] 2024-08-08T21:30:12.1040986Z test_decomp.py::TestDecompCUDA::test_quick_rsub_cuda_uint8 PASSED [0.1567s] [ 70%] 2024-08-08T21:30:12.1042172Z test_decomp.py::TestDecompCUDA::test_quick_select_cuda_int16 SKIPPED [0.0048s] (only backwards is decomposed, but dtype doesn't support AD) [ 71%] 2024-08-08T21:30:12.1042893Z test_decomp.py::TestDecompCUDA::test_quick_sgn_cuda_complex128 PASSED [0.0931s] [ 71%] 2024-08-08T21:30:12.1043547Z test_decomp.py::TestDecompCUDA::test_quick_sgn_cuda_int16 PASSED [0.0388s] [ 71%] 2024-08-08T21:30:12.1044204Z test_decomp.py::TestDecompCUDA::test_quick_sigmoid_cuda_bool PASSED [0.0151s] [ 72%] 2024-08-08T21:30:12.1044894Z test_decomp.py::TestDecompCUDA::test_quick_sigmoid_cuda_complex64 PASSED [1.0076s] [ 72%] 2024-08-08T21:30:12.1045570Z test_decomp.py::TestDecompCUDA::test_quick_sigmoid_cuda_int8 PASSED [0.0147s] [ 72%] 2024-08-08T21:30:12.1046226Z test_decomp.py::TestDecompCUDA::test_quick_sign_cuda_float16 PASSED [0.0066s] [ 72%] 2024-08-08T21:30:12.1047169Z test_decomp.py::TestDecompCUDA::test_quick_sign_cuda_int32 PASSED [0.0389s] [ 73%] 2024-08-08T21:30:12.1048054Z test_decomp.py::TestDecompCUDA::test_quick_sign_cuda_uint8 PASSED [0.0390s] [ 73%] 2024-08-08T21:30:12.1048722Z test_decomp.py::TestDecompCUDA::test_quick_signbit_cuda_bool PASSED [0.0388s] [ 73%] 2024-08-08T21:30:12.1049404Z test_decomp.py::TestDecompCUDA::test_quick_sin_cuda_complex32 PASSED [0.8329s] [ 74%] 2024-08-08T21:30:12.1050037Z test_decomp.py::TestDecompCUDA::test_quick_sin_cuda_int64 PASSED [0.0395s] [ 74%] 2024-08-08T21:30:12.1050658Z test_decomp.py::TestDecompCUDA::test_quick_sin_cuda_int8 PASSED [0.0400s] [ 74%] 2024-08-08T21:30:12.1051335Z test_decomp.py::TestDecompCUDA::test_quick_sinc_cuda_bfloat16 PASSED [0.2180s] [ 75%] 2024-08-08T21:30:12.1052717Z test_decomp.py::TestDecompCUDA::test_quick_sinc_cuda_bool PASSED [0.0149s] [ 75%] 2024-08-08T21:30:12.1053485Z test_decomp.py::TestDecompCUDA::test_quick_sinh_cuda_complex64 PASSED [0.4531s] [ 75%] 2024-08-08T21:30:12.1054219Z test_decomp.py::TestDecompCUDA::test_quick_slice_cuda_float16 PASSED [0.0106s] [ 75%] 2024-08-08T21:30:12.1054877Z test_decomp.py::TestDecompCUDA::test_quick_slice_cuda_float32 PASSED [0.1541s] [ 76%] 2024-08-08T21:30:12.1055542Z test_decomp.py::TestDecompCUDA::test_quick_slice_cuda_int16 PASSED [0.1225s] [ 76%] 2024-08-08T21:30:12.1056237Z test_decomp.py::TestDecompCUDA::test_quick_slice_scatter_cuda_int64 PASSED [0.6706s] [ 76%] 2024-08-08T21:30:12.1056951Z test_decomp.py::TestDecompCUDA::test_quick_special_entr_cuda_bfloat16 PASSED [0.2290s] [ 77%] 2024-08-08T21:30:12.1057661Z test_decomp.py::TestDecompCUDA::test_quick_special_entr_cuda_bool PASSED [0.0135s] [ 77%] 2024-08-08T21:30:12.1058676Z test_decomp.py::TestDecompCUDA::test_quick_special_entr_cuda_int64 PASSED [0.0114s] [ 77%] 2024-08-08T21:30:12.1059539Z test_decomp.py::TestDecompCUDA::test_quick_special_i0e_cuda_float64 PASSED [0.0149s] [ 78%] 2024-08-08T21:30:12.1060230Z test_decomp.py::TestDecompCUDA::test_quick_special_i0e_cuda_uint8 PASSED [0.0127s] [ 78%] 2024-08-08T21:30:12.1060911Z test_decomp.py::TestDecompCUDA::test_quick_special_i1_cuda_int64 PASSED [0.0126s] [ 78%] 2024-08-08T21:30:12.1061665Z test_decomp.py::TestDecompCUDA::test_quick_special_log_ndtr_cuda_float32 PASSED [0.0216s] [ 79%] 2024-08-08T21:30:12.1062395Z test_decomp.py::TestDecompCUDA::test_quick_special_log_ndtr_cuda_float64 PASSED [0.5119s] [ 79%] 2024-08-08T21:30:12.1063111Z test_decomp.py::TestDecompCUDA::test_quick_special_log_ndtr_cuda_uint8 PASSED [0.0151s] [ 79%] 2024-08-08T21:30:12.1064282Z test_decomp.py::TestDecompCUDA::test_quick_special_ndtr_cuda_float32 SKIPPED [0.0026s] (special.ndtr in torch.float32 not supported) [ 79%] 2024-08-08T21:30:12.1065406Z test_decomp.py::TestDecompCUDA::test_quick_special_ndtr_cuda_float64 SKIPPED [0.0030s] (special.ndtr in torch.float64 not supported) [ 80%] 2024-08-08T21:30:12.1066142Z test_decomp.py::TestDecompCUDA::test_quick_special_zeta_cuda_float64 PASSED [2.2258s] [ 80%] 2024-08-08T21:30:12.1066822Z test_decomp.py::TestDecompCUDA::test_quick_split_cuda_complex64 PASSED [0.1477s] [ 80%] 2024-08-08T21:30:12.1067544Z test_decomp.py::TestDecompCUDA::test_quick_split_list_args_cuda_float32 PASSED [0.1902s] [ 81%] 2024-08-08T21:30:12.1068275Z test_decomp.py::TestDecompCUDA::test_quick_split_list_args_cuda_int64 PASSED [0.1403s] [ 81%] 2024-08-08T21:30:12.1069072Z test_decomp.py::TestDecompCUDA::test_quick_split_with_sizes_copy_cuda_complex128 PASSED [0.4213s] [ 81%] 2024-08-08T21:30:12.1069868Z test_decomp.py::TestDecompCUDA::test_quick_split_with_sizes_copy_cuda_float32 PASSED [0.2534s] [ 82%] 2024-08-08T21:30:12.1070593Z test_decomp.py::TestDecompCUDA::test_quick_split_with_sizes_cuda_uint8 PASSED [0.1851s] [ 82%] 2024-08-08T21:30:12.1071248Z test_decomp.py::TestDecompCUDA::test_quick_sqrt_cuda_float16 PASSED [0.0071s] [ 82%] 2024-08-08T21:30:12.1071914Z test_decomp.py::TestDecompCUDA::test_quick_sqrt_cuda_float64 PASSED [0.0507s] [ 82%] 2024-08-08T21:30:12.1072607Z test_decomp.py::TestDecompCUDA::test_quick_squeeze_cuda_complex32 PASSED [0.1091s] [ 83%] 2024-08-08T21:30:12.1073379Z test_decomp.py::TestDecompCUDA::test_quick_squeeze_multiple_cuda_complex32 PASSED [0.0884s] [ 83%] 2024-08-08T21:30:12.1074026Z test_decomp.py::TestDecompCUDA::test_quick_std_cuda_float32 PASSED [0.2086s] [ 83%] 2024-08-08T21:30:12.1074776Z test_decomp.py::TestDecompCUDA::test_quick_std_mean_cuda_bfloat16 PASSED [0.0314s] [ 84%] 2024-08-08T21:30:12.1075622Z test_decomp.py::TestDecompCUDA::test_quick_std_mean_unbiased_cuda_complex64 PASSED [0.0457s] [ 84%] 2024-08-08T21:30:12.1076260Z test_decomp.py::TestDecompCUDA::test_quick_sub_cuda_int16 PASSED [0.1643s] [ 84%] 2024-08-08T21:30:12.1076889Z test_decomp.py::TestDecompCUDA::test_quick_sub_cuda_int32 PASSED [0.1616s] [ 85%] 2024-08-08T21:30:12.1077575Z test_decomp.py::TestDecompCUDA::test_quick_sum_cuda_complex128 PASSED [0.1808s] [ 85%] 2024-08-08T21:30:12.1078230Z test_decomp.py::TestDecompCUDA::test_quick_t_copy_cuda_int64 PASSED [0.0110s] [ 85%] 2024-08-08T21:30:12.1078893Z test_decomp.py::TestDecompCUDA::test_quick_t_copy_cuda_uint8 PASSED [0.0103s] [ 86%] 2024-08-08T21:30:12.1079538Z test_decomp.py::TestDecompCUDA::test_quick_t_cuda_bfloat16 PASSED [0.0091s] [ 86%] 2024-08-08T21:30:12.1080182Z test_decomp.py::TestDecompCUDA::test_quick_t_cuda_float64 PASSED [0.0170s] [ 86%] 2024-08-08T21:30:12.1080823Z test_decomp.py::TestDecompCUDA::test_quick_t_cuda_int64 PASSED [0.0099s] [ 86%] 2024-08-08T21:30:12.1081487Z test_decomp.py::TestDecompCUDA::test_quick_take_cuda_bfloat16 PASSED [0.0189s] [ 87%] 2024-08-08T21:30:12.1082150Z test_decomp.py::TestDecompCUDA::test_quick_take_cuda_float64 PASSED [0.0511s] [ 87%] 2024-08-08T21:30:12.1082816Z test_decomp.py::TestDecompCUDA::test_quick_tan_cuda_complex128 PASSED [0.5826s] [ 87%] 2024-08-08T21:30:12.1083454Z test_decomp.py::TestDecompCUDA::test_quick_tanh_cuda_uint8 PASSED [0.0406s] [ 88%] 2024-08-08T21:30:12.1084158Z test_decomp.py::TestDecompCUDA::test_quick_transpose_cuda_float64 PASSED [0.0693s] [ 88%] 2024-08-08T21:30:12.1084828Z test_decomp.py::TestDecompCUDA::test_quick_tril_cuda_complex64 PASSED [0.3174s] [ 88%] 2024-08-08T21:30:12.1085579Z test_decomp.py::TestDecompCUDA::test_quick_tril_indices_cuda_int64 PASSED [0.0152s] [ 89%] 2024-08-08T21:30:12.1086275Z test_decomp.py::TestDecompCUDA::test_quick_triu_cuda_bool PASSED [0.1384s] [ 89%] 2024-08-08T21:30:12.1086945Z test_decomp.py::TestDecompCUDA::test_quick_triu_cuda_complex64 PASSED [0.3182s] [ 89%] 2024-08-08T21:30:12.1087598Z test_decomp.py::TestDecompCUDA::test_quick_triu_cuda_int16 PASSED [0.1387s] [ 89%] 2024-08-08T21:30:12.1088228Z test_decomp.py::TestDecompCUDA::test_quick_triu_cuda_int64 PASSED [0.1384s] [ 90%] 2024-08-08T21:30:12.1088918Z test_decomp.py::TestDecompCUDA::test_quick_triu_indices_cuda_int64 PASSED [0.0147s] [ 90%] 2024-08-08T21:30:12.1089582Z test_decomp.py::TestDecompCUDA::test_quick_trunc_cuda_int16 PASSED [0.0393s] [ 90%] 2024-08-08T21:30:12.1090241Z test_decomp.py::TestDecompCUDA::test_quick_unbind_cuda_int16 PASSED [0.1472s] [ 91%] 2024-08-08T21:30:12.1090972Z test_decomp.py::TestDecompCUDA::test_quick_unfold_copy_cuda_complex32 PASSED [2.4285s] [ 91%] 2024-08-08T21:30:12.1091684Z test_decomp.py::TestDecompCUDA::test_quick_unfold_cuda_complex128 PASSED [2.3526s] [ 91%] 2024-08-08T21:30:12.1092395Z test_decomp.py::TestDecompCUDA::test_quick_unsafe_split_cuda_bfloat16 PASSED [0.0086s] [ 92%] 2024-08-08T21:30:12.1093129Z test_decomp.py::TestDecompCUDA::test_quick_unsafe_split_cuda_complex32 PASSED [0.3184s] [ 92%] 2024-08-08T21:30:12.1093818Z test_decomp.py::TestDecompCUDA::test_quick_unsafe_split_cuda_int8 PASSED [0.0654s] [ 92%] 2024-08-08T21:30:12.1094526Z test_decomp.py::TestDecompCUDA::test_quick_unsqueeze_copy_cuda_int32 PASSED [0.0543s] [ 93%] 2024-08-08T21:30:12.1095261Z test_decomp.py::TestDecompCUDA::test_quick_unsqueeze_cuda_bool PASSED [0.0540s] [ 93%] 2024-08-08T21:30:12.1095989Z test_decomp.py::TestDecompCUDA::test_quick_unsqueeze_cuda_uint8 PASSED [0.0537s] [ 93%] 2024-08-08T21:30:12.1096715Z test_decomp.py::TestDecompCUDA::test_quick_var_unbiased_cuda_float16 PASSED [0.0095s] [ 93%] 2024-08-08T21:30:12.1097381Z test_decomp.py::TestDecompCUDA::test_quick_vdot_cuda_bfloat16 PASSED [0.0069s] [ 94%] 2024-08-08T21:30:12.1098052Z test_decomp.py::TestDecompCUDA::test_quick_view_copy_cuda_bool PASSED [0.0867s] [ 94%] 2024-08-08T21:30:12.1098755Z test_decomp.py::TestDecompCUDA::test_quick_view_copy_cuda_float64 PASSED [0.1182s] [ 94%] 2024-08-08T21:30:12.1099428Z test_decomp.py::TestDecompCUDA::test_quick_view_copy_cuda_int16 PASSED [0.0871s] [ 95%] 2024-08-08T21:30:12.1100094Z test_decomp.py::TestDecompCUDA::test_quick_view_cuda_bfloat16 PASSED [0.0152s] [ 95%] 2024-08-08T21:30:12.1100773Z test_decomp.py::TestDecompCUDA::test_quick_view_cuda_complex32 PASSED [0.1988s] [ 95%] 2024-08-08T21:30:12.1101423Z test_decomp.py::TestDecompCUDA::test_quick_view_cuda_int64 PASSED [0.0871s] [ 96%] 2024-08-08T21:30:12.1102077Z test_decomp.py::TestDecompCUDA::test_quick_view_cuda_int8 PASSED [0.0865s] [ 96%] 2024-08-08T21:30:12.1102721Z test_decomp.py::TestDecompCUDA::test_quick_view_cuda_uint8 PASSED [0.0855s] [ 96%] 2024-08-08T21:30:12.1103363Z test_decomp.py::TestDecompCUDA::test_quick_where_cuda_int64 PASSED [0.1422s] [ 96%] 2024-08-08T21:30:12.1104016Z test_decomp.py::TestDecompCUDA::test_quick_where_cuda_int8 PASSED [0.1417s] [ 97%] 2024-08-08T21:30:12.1104661Z test_decomp.py::TestDecompCUDA::test_quick_xlogy_cuda_int32 PASSED [0.1637s] [ 97%] 2024-08-08T21:30:12.1105653Z test_decomp.py::TestDecompCUDA::test_quick_zero__cuda_complex64 SKIPPED [0.0027s] (zero_ in torch.complex64 not supported) [ 97%] 2024-08-08T21:30:12.1106346Z test_decomp.py::TestDecompCUDA::test_quick_zeros_cuda_bool PASSED [0.0069s] [ 98%] 2024-08-08T21:30:12.1107037Z test_decomp.py::TestDecompCUDA::test_quick_zeros_cuda_int16 PASSED [0.0074s] [ 98%] 2024-08-08T21:30:12.1107694Z test_decomp.py::TestDecompCUDA::test_quick_zeros_cuda_int64 PASSED [0.0069s] [ 98%] 2024-08-08T21:30:12.1108373Z test_decomp.py::TestDecompCUDA::test_quick_zeros_like_cuda_int64 PASSED [0.0233s] [ 99%] 2024-08-08T21:30:12.1109047Z test_decomp.py::TestDecompCUDA::test_quick_zeros_like_cuda_int8 PASSED [0.0235s] [ 99%] 2024-08-08T21:30:12.1109694Z test_decomp.py::TestDecompCUDA::test_uniform_cuda PASSED [0.0028s] [ 99%] 2024-08-08T21:30:12.1110448Z test_decomp.py::DecompOneOffTestsCUDA::test_threshold_backward_dtype_cuda PASSED [0.0108s] [100%] 2024-08-08T21:30:12.1110471Z 2024-08-08T21:30:12.1111455Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_decomp/test_decomp-32ed1af9f4a2f5b1.xml - 2024-08-08T21:30:12.1112323Z ==== 285 passed, 36 skipped, 899 deselected, 8 xfailed in 74.26s (0:01:14) ===== 2024-08-08T21:30:12.1113649Z The following tests failed and then succeeded when run in a new process['test/test_decomp.py::TestDecompCUDA::test_quick_core_backward_split_with_sizes_copy_cuda_float64'] 2024-08-08T21:30:12.1113689Z 2024-08-08T21:30:12.1114396Z FINISHED PRINTING LOG FILE of test_decomp 1/7 (test/test-reports/test_decomp_1.7_e987e43d390a110b_.log) 2024-08-08T21:30:12.1114415Z 2024-08-08T21:30:15.6151326Z Running test_ops_jit 2/2 ... [2024-08-08 21:30:15.614575] 2024-08-08T21:30:15.6156266Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_ops_jit.py', '-m', 'not serial', '--shard-id=2', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:30:15.615124] 2024-08-08T21:45:19.5422176Z 2024-08-08T21:45:19.5424984Z test_ops_jit 2/2 was successful, full logs can be found in artifacts with path test/test-reports/test_ops_jit_2.2_50eb32c2d3d3bd56_.log 2024-08-08T21:45:19.5792841Z Running 587 items in this shard: test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_acos_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_asinh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_atan_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_atanh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_div_floor_rounding_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_erf_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_erfc_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_exp2_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_expm1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_ge_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_gt_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_igammac_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_linalg_det_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_linalg_inv_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_log_softmax_with_dtype_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_logit_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_logsumexp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_lt_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_mH_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_matrix_exp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_max_binary_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_mvlgamma_mvlgamma_p_3_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_neg_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_nn_functional_conv2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_nn_functional_conv3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_outer_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_round_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_round_decimals_0_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_round_decimals_neg_3_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_transpose_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_trunc_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_H_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_T_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_T_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___getitem___cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___radd___cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___radd___cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rdiv___cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rmod___cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rmul___cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rmul___cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rsub___cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rsub___cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__batch_norm_with_update_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__chunk_cat_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__native_batch_norm_legit_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__segment_reduce_lengths_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__segment_reduce_offsets_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__softmax_backward_data_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__unsafe_masked_index_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__unsafe_masked_index_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__unsafe_masked_index_put_accumulate_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__upsample_bilinear2d_aa_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_abs_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_abs_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_acos_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_acos_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_add_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addbmm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addcmul_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addmm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addmm_decomposed_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addmv_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addr_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addr_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_alias_copy_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_alias_copy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_all_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_all_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_allclose_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_allclose_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_amin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_angle_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_any_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_any_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_arange_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_argmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_copy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_partial_views_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_partial_views_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_scatter_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_scatter_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_asinh_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atan_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atanh_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atanh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atleast_1d_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atleast_1d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atleast_2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bfloat16_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bfloat16_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_block_diag_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bmm_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bmm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bool_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_broadcast_tensors_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_broadcast_to_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_byte_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cartesian_prod_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cdist_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cdouble_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ceil_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_chalf_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_chalf_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_char_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_inverse_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_solve_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_chunk_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_chunk_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_clamp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_clamp_min_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_column_stack_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_combinations_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_complex_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_conj_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_conj_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_conj_physical_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_contiguous_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cummin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cumprod_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cumsum_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cumulative_trapezoid_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_deg2rad_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diag_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diag_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagflat_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagonal_copy_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagonal_copy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagonal_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagonal_scatter_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagonal_scatter_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diff_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diff_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_dist_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_dist_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_div_floor_rounding_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_div_trunc_rounding_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_double_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_dstack_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_einsum_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_einsum_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_empty_like_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_empty_permuted_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_equal_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_erfc_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_erfinv_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_exp2_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_exp2_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_exp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_expand_as_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_expand_copy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_expand_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_expm1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_eye_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_fft2_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_fft_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_fft_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_fftn_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_hfft2_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_ifft_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_ifftn_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_ifftn_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_ihfft2_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_ihfft_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_irfft2_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_irfft_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_irfftn_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_rfft_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_rfftn_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fill_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fill_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_flatten_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_flip_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fliplr_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_flipud_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_float_power_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_floor_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_floor_divide_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_frac_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_frexp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_full_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_full_like_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_full_like_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_gather_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_gather_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ge_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_geometric_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_geqrf_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_geqrf_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_half_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_heaviside_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_hstack_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_hstack_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_hypot_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_igamma_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_add_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_add_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_copy_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_copy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_put_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_put_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_reduce_amax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_reduce_amin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_reduce_prod_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_select_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_select_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_inner_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isclose_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isfinite_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isinf_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isneginf_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isposinf_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isreal_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isreal_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_istft_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_item_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_item_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_jiterator_2inputs_2outputs_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_jiterator_4inputs_with_extra_args_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_jiterator_4inputs_with_extra_args_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_jiterator_binary_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_jiterator_unary_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_kron_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_kron_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ldexp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_le_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_lerp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cholesky_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cholesky_ex_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cholesky_ex_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cond_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cross_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cross_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_det_singular_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_diagonal_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_diagonal_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_eig_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_eigvals_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_eigvalsh_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_eigvalsh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_householder_product_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_inv_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_ldl_factor_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_ldl_factor_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_ldl_solve_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lstsq_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lstsq_grad_oriented_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lstsq_grad_oriented_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lu_factor_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lu_factor_ex_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lu_solve_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lu_solve_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_matrix_norm_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_matrix_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_matrix_power_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_matrix_rank_hermitian_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_multi_dot_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_multi_dot_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_norm_subgradients_at_zero_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_pinv_hermitian_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_pinv_singular_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_pinv_singular_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_qr_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_solve_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_solve_ex_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_solve_triangular_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_solve_triangular_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_svd_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_svdvals_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_tensorinv_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_tensorinv_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_vander_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_vander_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_vecdot_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linspace_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linspace_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linspace_tensor_overload_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linspace_tensor_overload_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_log1p_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_log_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_log_softmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logaddexp2_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logcumsumexp_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logdet_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logical_and_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logical_not_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logical_or_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logical_xor_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logical_xor_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logit_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logspace_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logspace_tensor_overload_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_long_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_long_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_lu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_lu_solve_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_lu_solve_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_lu_unpack_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mH_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mH_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mT_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mT_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_amax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_argmin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_cumsum_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_cumsum_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_fill_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_fill_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_log_softmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_logaddexp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_mean_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_mean_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_prod_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_prod_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_select_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_select_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_softmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_softmin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_std_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_std_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_sum_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_var_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_matmul_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_matmul_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_matrix_exp_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_matrix_exp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_max_binary_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_maximum_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_median_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_meshgrid_list_of_tensors_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_meshgrid_variadic_tensors_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_min_reduction_no_dim_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_min_reduction_with_dim_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mode_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_multinomial_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mvlgamma_mvlgamma_p_3_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nanmean_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nanmedian_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nansum_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_narrow_copy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_native_batch_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_native_dropout_backward_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_new_empty_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_new_ones_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_new_ones_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_new_zeros_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_new_zeros_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nextafter_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_adaptive_avg_pool2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_adaptive_max_pool1d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_adaptive_max_pool2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_adaptive_max_pool3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_avg_pool2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_batch_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_batch_norm_without_cudnn_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_bilinear_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_binary_cross_entropy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_binary_cross_entropy_with_logits_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_celu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_channel_shuffle_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv1d_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv1d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv_transpose1d_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv_transpose1d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv_transpose2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv_transpose3d_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_cosine_embedding_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_ctc_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_dropout2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_dropout3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_embedding_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_feature_alpha_dropout_without_train_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_fractional_max_pool2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_fractional_max_pool3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_gelu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_glu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_grid_sample_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_hardshrink_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_hinge_embedding_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_interpolate_area_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_interpolate_linear_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_interpolate_nearest-exact_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_interpolate_nearest_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_interpolate_trilinear_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_kl_div_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_layer_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_leaky_relu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_linear_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_logsigmoid_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_margin_ranking_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_pool1d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_pool2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_unpool1d_grad_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_unpool2d_grad_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_unpool3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_mish_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_mse_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_multi_head_attention_forward_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_multi_margin_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_multilabel_margin_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_multilabel_soft_margin_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_nll_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pad_circular_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pad_constant_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pad_replicate_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pairwise_distance_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pairwise_distance_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pixel_unshuffle_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_relu6_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_relu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_rms_norm_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_rms_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_silu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_soft_margin_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_softsign_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_tanhshrink_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_unfold_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_upsample_bilinear_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nonzero_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nonzero_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nonzero_static_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_norm_fro_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_norm_nuc_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_norm_nuc_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_normal_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_normal_in_place_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ones_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ones_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ones_like_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ormqr_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_outer_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_outer_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_pca_lowrank_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_pca_lowrank_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_permute_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_pinverse_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_polygamma_polygamma_n_4_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_positive_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_positive_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_pow_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_prod_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_qr_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_quantile_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_randint_like_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ravel_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ravel_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_real_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_reciprocal_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_remainder_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_renorm_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_renorm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_repeat_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_repeat_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_repeat_interleave_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_repeat_interleave_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_reshape_as_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_reshape_as_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_reshape_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_resize_as__cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_resolve_conj_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_resolve_neg_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_resolve_neg_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_roll_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_rot90_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_rot90_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_round_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_round_decimals_neg_3_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_rsqrt_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scalar_tensor_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_add_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_reduce_amax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_reduce_amin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_reduce_mean_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_select_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_select_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sgn_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sgn_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_short_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sigmoid_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_signal_windows_exponential_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_signal_windows_general_cosine_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_signal_windows_general_hamming_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_signal_windows_hamming_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sin_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sinc_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sinh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_slice_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_slice_scatter_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_softmax_with_dtype_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sparse_sampled_addmm_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sparse_sampled_addmm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_bessel_j1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_bessel_y1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_chebyshev_polynomial_t_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_chebyshev_polynomial_v_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_chebyshev_polynomial_w_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_hermite_polynomial_he_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_i1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_laguerre_polynomial_l_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_log_ndtr_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_modified_bessel_i0_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_ndtri_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_polygamma_special_polygamma_n_0_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_scaled_modified_bessel_k0_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_shifted_chebyshev_polynomial_u_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_shifted_chebyshev_polynomial_v_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_shifted_chebyshev_polynomial_w_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_spherical_bessel_j0_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_zeta_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_list_args_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_list_args_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_with_sizes_copy_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_with_sizes_copy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_with_sizes_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sqrt_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_square_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_stack_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_stack_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_std_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_std_mean_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_std_unbiased_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_std_unbiased_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sub_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sub_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sum_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sum_to_size_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_svd_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_svd_lowrank_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_svd_lowrank_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_t_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_take_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_take_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_tanh_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_tanh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_tensor_split_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_topk_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_torch_ops_aten__efficient_attention_forward_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_torch_ops_aten__safe_softmax_default_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_trace_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_trace_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_transpose_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_trapezoid_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_trapz_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_triangular_solve_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_tril_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_triu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_true_divide_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_trunc_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unbind_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unbind_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unflatten_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_uniform_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_uniform_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unique_consecutive_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unique_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unsafe_chunk_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unsafe_chunk_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unsqueeze_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unsqueeze_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_var_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_var_unbiased_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_as_complex_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_as_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_as_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_as_real_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_copy_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_copy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_vsplit_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_where_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_zeros_cuda_complex64 2024-08-08T21:45:19.6103708Z 2024-08-08T21:45:23.3686979Z Running inductor/test_aot_inductor 4/9 ... [2024-08-08 21:45:23.368048] 2024-08-08T21:45:23.3690855Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_aot_inductor.py', '-m', 'not serial', '--shard-id=4', '--num-shards=9', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:45:23.368538] 2024-08-08T21:45:32.0044388Z 2024-08-08T21:45:32.0047884Z inductor/test_aot_inductor 4/9 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_aot_inductor_4.9_c757054a6ba12aa7_.log 2024-08-08T21:45:32.0050165Z 2024-08-08T21:45:35.9418552Z Running test_mkldnn_verbose 1/1 ... [2024-08-08 21:45:35.941268] 2024-08-08T21:45:35.9423369Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_mkldnn_verbose.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:45:35.941792] 2024-08-08T21:45:44.5221244Z 2024-08-08T21:45:44.5223167Z test_mkldnn_verbose 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_mkldnn_verbose_1.1_7fa10cd3eaf682b7_.log 2024-08-08T21:45:44.5225287Z Running 2 items in this shard: test/test_mkldnn_verbose.py::TestMKLDNNVerbose::test_verbose_off, test/test_mkldnn_verbose.py::TestMKLDNNVerbose::test_verbose_on 2024-08-08T21:45:44.5226400Z 2024-08-08T21:45:48.3832464Z Running test_expanded_weights 1/1 ... [2024-08-08 21:45:48.382613] 2024-08-08T21:45:48.3836735Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_expanded_weights.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:45:48.383132] 2024-08-08T21:46:11.6383673Z 2024-08-08T21:46:11.6386279Z test_expanded_weights 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_expanded_weights_1.1_42e061686ac92257_.log 2024-08-08T21:46:11.6543721Z Running 220 items in this shard: test/test_expanded_weights.py::TestExpandedWeightHelperFunctionCUDA::test_forward_helper_cuda, test/test_expanded_weights.py::TestExpandedWeightHelperFunctionCUDA::test_forward_helper_failure_args_cuda, test/test_expanded_weights.py::TestExpandedWeightHelperFunctionCUDA::test_set_grad_sample_if_exists_cuda, test/test_expanded_weights.py::TestExpandedWeightHelperFunctionCUDA::test_set_grad_sample_if_exists_failure_cuda, test/test_expanded_weights.py::TestExpandedWeightHelperFunctionCUDA::test_sum_over_all_but_batch_and_last_n_cuda, test/test_expanded_weights.py::TestExpandedWeightHelperFunctionCUDA::test_unpack_expanded_weight_or_tensor_cuda, test/test_expanded_weights.py::TestExpandedWeightHelperFunctionCUDA::test_unpack_expanded_weight_or_tensor_failure_cuda, test/test_expanded_weights.py::TestExpandedWeightHelperFunctionCUDA::test_unpack_expanded_weight_or_tensor_with_custom_function_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_cnn_model_mean_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_cnn_model_sum_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_embedding_model_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_error_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv1d_cuda_bfloat16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv1d_cuda_complex128, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv1d_cuda_complex32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv1d_cuda_complex64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv1d_cuda_float16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv1d_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv1d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv2d_cuda_bfloat16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv2d_cuda_complex128, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv2d_cuda_complex32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv2d_cuda_complex64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv2d_cuda_float16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv2d_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv2d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv3d_cuda_bfloat16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv3d_cuda_complex128, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv3d_cuda_complex32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv3d_cuda_complex64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv3d_cuda_float16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv3d_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_conv3d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_embedding_cuda_bfloat16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_embedding_cuda_float16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_embedding_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_embedding_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_group_norm_cuda_bfloat16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_group_norm_cuda_float16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_group_norm_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_group_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_instance_norm_cuda_bfloat16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_instance_norm_cuda_float16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_instance_norm_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_instance_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_layer_norm_cuda_bfloat16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_layer_norm_cuda_float16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_layer_norm_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_layer_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_linear_cuda_bfloat16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_linear_cuda_complex128, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_linear_cuda_complex64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_linear_cuda_float16, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_linear_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_forward_nn_functional_linear_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_mean_nn_functional_conv1d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_mean_nn_functional_conv2d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_mean_nn_functional_conv3d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_mean_nn_functional_embedding_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_mean_nn_functional_group_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_mean_nn_functional_instance_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_mean_nn_functional_layer_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_mean_nn_functional_linear_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_sum_nn_functional_conv1d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_sum_nn_functional_conv2d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_sum_nn_functional_conv3d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_sum_nn_functional_embedding_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_sum_nn_functional_group_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_sum_nn_functional_instance_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_sum_nn_functional_layer_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weight_per_sample_grad_sum_nn_functional_linear_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weights_per_sample_grad_input_no_grad_nn_functional_conv1d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weights_per_sample_grad_input_no_grad_nn_functional_conv2d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weights_per_sample_grad_input_no_grad_nn_functional_conv3d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weights_per_sample_grad_input_no_grad_nn_functional_embedding_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weights_per_sample_grad_input_no_grad_nn_functional_group_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weights_per_sample_grad_input_no_grad_nn_functional_instance_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weights_per_sample_grad_input_no_grad_nn_functional_layer_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_expanded_weights_per_sample_grad_input_no_grad_nn_functional_linear_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_group_norm_error_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_group_norm_model_num_dim_1_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_group_norm_model_num_dim_2_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_group_norm_model_num_dim_3_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_instance_norm_model_num_dim_1_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_instance_norm_model_num_dim_2_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_instance_norm_model_num_dim_3_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_layer_norm_model_num_dim_1_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_layer_norm_model_num_dim_2_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_layer_norm_model_num_dim_3_cuda, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_unsupported_expand_weights_nn_functional_conv1d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_unsupported_expand_weights_nn_functional_conv2d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_unsupported_expand_weights_nn_functional_conv3d_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_unsupported_expand_weights_nn_functional_embedding_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_unsupported_expand_weights_nn_functional_group_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_unsupported_expand_weights_nn_functional_instance_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_unsupported_expand_weights_nn_functional_layer_norm_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightFunctionalCUDA::test_unsupported_expand_weights_nn_functional_linear_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_circular_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_circular_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_circular_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad1_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad1_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad1_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad1size1_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad1size1_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad1size1_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad2size1_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad2size1_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_pad2size1_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_reflect_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_reflect_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_reflect_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_replicate_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_replicate_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_replicate_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_stride_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_stride_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_stride_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_zero_batch_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_zero_batch_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_zero_batch_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_zeros_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_zeros_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv1d_zeros_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_circular_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_circular_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_circular_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_dilated_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_dilated_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_dilated_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_no_bias_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_no_bias_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_no_bias_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_padding_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_padding_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_padding_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_reflect_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_reflect_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_reflect_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_replicate_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_replicate_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_replicate_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_strided_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_strided_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_strided_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_zero_batch_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_zero_batch_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_zero_batch_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_zeros_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_zeros_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv2d_zeros_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_1x1x1_no_bias_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_1x1x1_no_bias_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_1x1x1_no_bias_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_circular_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_circular_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_circular_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_no_bias_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_no_bias_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_no_bias_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_replicate_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_replicate_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_replicate_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_stride_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_stride_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_stride_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_stride_padding_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_stride_padding_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_stride_padding_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_zero_batch_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_zero_batch_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_zero_batch_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_zeros_stride2_pad2_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_zeros_stride2_pad2_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Conv3d_zeros_stride2_pad2_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Embedding_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Embedding_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Embedding_discontiguous_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Embedding_discontiguous_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Embedding_discontiguous_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Embedding_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_LayerNorm_3d_no_affine_large_feature_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_LayerNorm_3d_no_affine_large_feature_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_LayerNorm_3d_no_affine_large_feature_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Linear_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Linear_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Linear_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Linear_no_batch_dim_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Linear_no_batch_dim_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Linear_no_batch_dim_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Linear_no_bias_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Linear_no_bias_cuda_double_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_Linear_no_bias_multiple_inputs_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_GRU_eval_mode_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_GRU_eval_mode_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_GRU_train_mode_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_GRU_train_mode_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_LSTM_eval_mode_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_LSTM_eval_mode_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_LSTM_train_mode_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_LSTM_train_mode_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_RNN_eval_mode_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_RNN_eval_mode_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_RNN_train_mode_cuda_float32, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_module_nn_RNN_train_mode_cuda_float64, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_per_sample_api_compute_batch_size_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_per_sample_api_compute_batch_size_not_pytreeable_cuda, test/test_expanded_weights.py::TestExpandedWeightModuleCUDA::test_per_sample_api_failing_cuda 2024-08-08T21:46:11.6698767Z 2024-08-08T21:46:15.4928173Z Running test_ops_fwd_gradients 1/2 ... [2024-08-08 21:46:15.492226] 2024-08-08T21:46:15.4932106Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_ops_fwd_gradients.py', '-m', 'not serial', '--shard-id=1', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:46:15.492694] 2024-08-08T21:57:47.6523447Z 2024-08-08T21:57:47.6525055Z test_ops 4/4 was successful, full logs can be found in artifacts with path test/test-reports/test_ops_4.4_830c00724824914e_.log 2024-08-08T21:57:48.1392845Z Running 8031 items in this shard: test/test_ops.py::TestSelfKwarg::test_self_kwargs, test/test_ops.py::TestCommonCUDA::test_compare_cpu_H_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu___rmatmul___cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu___rsub___cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__native_batch_norm_legit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs__conversions_cfloat_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs__conversions_complex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs__conversions_float_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs__conversions_long_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs__conversions_polar_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_alias_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_as_strided_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_as_strided_partial_views_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_atleast_2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_diagonal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_diagonal_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_dot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_dsplit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_exponential_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_flip_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_fmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_lerp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_linalg_diagonal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_linspace_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_log_normal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_logaddexp2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_logspace_tensor_overload_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_new_zeros_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_nn_functional_alpha_dropout_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_nn_functional_huber_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_nn_functional_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_nn_functional_relu6_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_nn_functional_softshrink_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_nn_functional_triplet_margin_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_select_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_t_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_t_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_true_divide_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_var_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_view_as_complex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_view_as_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__refs_vstack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__segment_reduce_lengths_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu__unsafe_masked_index_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_addmm_decomposed_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_as_strided_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_atleast_1d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_atleast_3d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_bitwise_right_shift_cuda_int64, test/test_ops.py::TestCommonCUDA::test_compare_cpu_block_diag_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_bool_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_cauchy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_cholesky_inverse_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_complex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_copysign_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_cross_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_dist_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_dot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_double_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_einsum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_empty_strided_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_fft_ifftshift_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_flip_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_grid_sampler_2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_index_fill_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_index_reduce_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_index_reduce_prod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_isin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_ldexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_linalg_det_singular_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_linalg_inv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_linalg_inv_ex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_linalg_ldl_factor_ex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_linalg_lu_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_linalg_matrix_rank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_linalg_vector_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_linspace_tensor_overload_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_log_softmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_logcumsumexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_logspace_tensor_overload_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_masked_normalize_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_masked_select_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_median_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_min_reduction_no_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_multinomial_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_narrow_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_native_batch_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_new_full_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_avg_pool1d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_batch_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_channel_shuffle_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_conv1d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_conv2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_cosine_similarity_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_ctc_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_dropout3d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_interpolate_trilinear_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_linear_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_local_response_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_max_unpool1d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_max_unpool2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_max_unpool3d_grad_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_multilabel_margin_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_pad_constant_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_soft_margin_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_softmin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nn_functional_upsample_bilinear_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nonzero_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_nonzero_static_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_norm_inf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_norm_nuc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_normal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_normal_number_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_ones_like_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_pca_lowrank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_polar_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_repeat_interleave_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_reshape_as_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_reshape_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_resize_as__cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_resolve_neg_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_rsub_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_scatter_add_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_select_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_sparse_sampled_addmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_special_shifted_chebyshev_polynomial_t_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_special_shifted_chebyshev_polynomial_w_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_split_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_split_with_sizes_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_stack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_take_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_triangular_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_unflatten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_unfold_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_unsqueeze_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_var_mean_unbiased_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_view_as_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_view_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_xlogy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_zero__cuda_float32, test/test_ops.py::TestCommonCUDA::test_compare_cpu_zeros_cuda_float32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_as_strided_partial_views_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_atleast_3d_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_bool_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_cfloat_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_chunk_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_clone_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_column_stack_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_conj_physical_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_diag_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_diagonal_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_dsplit_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_empty_permuted_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_eq_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_hstack_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_index_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_index_put_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_isfinite_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_item_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_log_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_mT_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_masked_fill_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_mul_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_narrow_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_narrow_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_new_empty_strided_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_new_ones_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_nn_functional_conv3d_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_nn_functional_conv_transpose2d_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_nonzero_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_permute_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_randn_like_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_real_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_reshape_as_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_resolve_neg_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_scalar_tensor_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_sigmoid_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_split_with_sizes_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_squeeze_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_squeeze_multiple_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_tan_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_true_divide_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_unbind_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_unfold_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_unsafe_split_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_complex_half_reference_testing_where_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_dtypes___rdiv___cuda, test/test_ops.py::TestCommonCUDA::test_dtypes___rmatmul___cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__batch_norm_with_update_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__chunk_cat_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs__conversions_char_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs__conversions_complex_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs__conversions_long_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_any_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_as_strided_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_asin_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_atan_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_bitwise_xor_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_broadcast_to_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_cauchy_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_clamp_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_contiguous_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_copysign_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_count_nonzero_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_cumsum_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_diagonal_copy_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_diagonal_scatter_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_digamma_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_div_no_rounding_mode_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_empty_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_erf_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_erfinv_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_exp2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_exp_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_exponential_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_eye_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_fft_ihfft2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_fft_ihfftn_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_fft_irfftn_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_fft_rfftn_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_flatten_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_flipud_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_floor_divide_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_geometric_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_imag_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_index_add_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_isclose_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_lcm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_logaddexp2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_logaddexp_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_logical_and_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_logspace_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_logspace_tensor_overload_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_lt_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_maximum_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_meshgrid_list_of_tensors_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_narrow_copy_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_narrow_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_new_ones_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_new_zeros_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_hinge_embedding_loss_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_log_softmax_with_dtype_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_margin_ranking_loss_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_mse_loss_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_pdist_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_poisson_nll_loss_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_relu6_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_softmax_with_dtype_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_softmin_with_dtype_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_tanhshrink_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_nn_functional_threshold_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_norm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_normal__in_place_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_positive_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_prod_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_rad2deg_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_randn_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_real_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_reshape_as_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_round_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_rsub_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_sgn_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_special_softmax_with_dtype_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_stft_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_trace_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_tril_indices_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_true_divide_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_unsqueeze_copy_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_var_mean_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_vdot_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_view_as_complex_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_view_as_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_view_copy_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_vsplit_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__refs_zeros_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__segment_reduce_offsets_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__unsafe_masked_index_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes__unsafe_masked_index_put_accumulate_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_abs_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_addmm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_allclose_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_amin_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_angle_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_argmax_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_argwhere_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_atan2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_baddbmm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_bincount_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_bitwise_and_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_bitwise_not_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_bitwise_or_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_bitwise_xor_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_broadcast_tensors_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_cdouble_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_cholesky_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_clamp_max_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_column_stack_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_conj_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_constant_pad_nd_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_contiguous_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_corrcoef_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_cumsum_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_deg2rad_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_diagflat_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_dist_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_div_trunc_rounding_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_double_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_dstack_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_empty_strided_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_eq_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_erfc_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_erfinv_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_exp2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_exp_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_exponential_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_eye_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_fft_hfft2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_fft_ifft2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_fft_ihfft_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_fft_irfft2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_fft_irfft_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_fft_rfft2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_flip_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_float_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_floor_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_gather_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_gcd_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_ge_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_heaviside_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_histogramdd_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_igamma_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_index_fill_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_index_reduce_amax_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_index_reduce_amin_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_index_select_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_inner_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_isposinf_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_item_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_jiterator_2inputs_2outputs_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_jiterator_4inputs_with_extra_args_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_le_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_cholesky_ex_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_diagonal_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_eigh_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_ldl_factor_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_ldl_solve_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_lstsq_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_norm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_pinv_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_pinv_hermitian_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_pinv_singular_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_qr_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linalg_slogdet_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_linspace_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_log10_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_log1p_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_log2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_log_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_log_normal_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_log_softmax_with_dtype_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_logical_or_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_logspace_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_logspace_tensor_overload_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_lu_unpack_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_mH_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_masked_amax_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_masked_argmax_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_masked_argmin_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_masked_normalize_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_masked_select_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_matmul_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_max_pool2d_with_indices_backward_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_maximum_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_median_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_meshgrid_list_of_tensors_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_mode_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_mul_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_multinomial_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_mv_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_mvlgamma_mvlgamma_p_3_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nan_to_num_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nanquantile_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_narrow_copy_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_ne_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_neg_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_new_empty_strided_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_adaptive_avg_pool2d_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_adaptive_max_pool1d_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_celu_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_conv_transpose1d_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_embedding_bag_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_group_norm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_hardshrink_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_layer_norm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_max_pool3d_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_max_unpool1d_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_max_unpool3d_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_multi_margin_loss_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_normalize_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_one_hot_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_pixel_shuffle_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_pixel_unshuffle_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_relu_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_rms_norm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_scaled_dot_product_attention_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_selu_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_soft_margin_loss_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_softmin_with_dtype_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nn_functional_upsample_nearest_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nonzero_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_nonzero_static_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_norm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_norm_fro_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_normal_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_normal_in_place_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_outer_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_polygamma_polygamma_n_0_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_polygamma_polygamma_n_2_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_polygamma_polygamma_n_3_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_positive_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_pow_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_prod_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_randint_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_randn_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_renorm_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_reshape_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_resize_as__cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_round_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_round_decimals_0_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_rsqrt_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_scatter_reduce_mean_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_select_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_sign_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_signal_windows_blackman_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_signal_windows_cosine_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_signal_windows_exponential_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_signal_windows_gaussian_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_signbit_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_slice_scatter_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_softmax_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_sparse_mm_reduce_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_special_bessel_y0_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_special_chebyshev_polynomial_t_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_special_chebyshev_polynomial_v_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_special_hermite_polynomial_he_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_special_ndtri_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_special_spherical_bessel_j0_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_split_with_sizes_copy_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_squeeze_multiple_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_stack_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_std_mean_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_std_unbiased_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_sum_to_size_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_svd_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_t_copy_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_take_along_dim_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_tan_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_tile_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_torch_ops_aten__safe_softmax_default_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_trace_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_trapezoid_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_triu_indices_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_unbind_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_unsafe_chunk_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_var_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_var_mean_unbiased_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_var_unbiased_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_view_copy_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_view_cuda, test/test_ops.py::TestCommonCUDA::test_dtypes_zeros_cuda, test/test_ops.py::TestCommonCUDA::test_errors_T_cuda, test/test_ops.py::TestCommonCUDA::test_errors___rand___cuda, test/test_ops.py::TestCommonCUDA::test_errors___rmod___cuda, test/test_ops.py::TestCommonCUDA::test_errors__chunk_cat_cuda, test/test_ops.py::TestCommonCUDA::test_errors_as_strided_scatter_cuda, test/test_ops.py::TestCommonCUDA::test_errors_bitwise_and_cuda, test/test_ops.py::TestCommonCUDA::test_errors_cat_cuda, test/test_ops.py::TestCommonCUDA::test_errors_cauchy_cuda, test/test_ops.py::TestCommonCUDA::test_errors_clamp_max_cuda, test/test_ops.py::TestCommonCUDA::test_errors_diff_cuda, test/test_ops.py::TestCommonCUDA::test_errors_div_floor_rounding_cuda, test/test_ops.py::TestCommonCUDA::test_errors_div_trunc_rounding_cuda, test/test_ops.py::TestCommonCUDA::test_errors_dsplit_cuda, test/test_ops.py::TestCommonCUDA::test_errors_fft_fftn_cuda, test/test_ops.py::TestCommonCUDA::test_errors_fft_hfft2_cuda, test/test_ops.py::TestCommonCUDA::test_errors_fft_ifft_cuda, test/test_ops.py::TestCommonCUDA::test_errors_fmax_cuda, test/test_ops.py::TestCommonCUDA::test_errors_gt_cuda, test/test_ops.py::TestCommonCUDA::test_errors_heaviside_cuda, test/test_ops.py::TestCommonCUDA::test_errors_jiterator_binary_return_by_ref_cuda, test/test_ops.py::TestCommonCUDA::test_errors_linalg_diagonal_cuda, test/test_ops.py::TestCommonCUDA::test_errors_linalg_lstsq_grad_oriented_cuda, test/test_ops.py::TestCommonCUDA::test_errors_log_normal_cuda, test/test_ops.py::TestCommonCUDA::test_errors_logaddexp_cuda, test/test_ops.py::TestCommonCUDA::test_errors_lt_cuda, test/test_ops.py::TestCommonCUDA::test_errors_nn_functional_adaptive_max_pool2d_cuda, test/test_ops.py::TestCommonCUDA::test_errors_nn_functional_avg_pool3d_cuda, test/test_ops.py::TestCommonCUDA::test_errors_nn_functional_conv1d_cuda, test/test_ops.py::TestCommonCUDA::test_errors_nn_functional_conv3d_cuda, test/test_ops.py::TestCommonCUDA::test_errors_nn_functional_max_pool3d_cuda, test/test_ops.py::TestCommonCUDA::test_errors_nn_functional_soft_margin_loss_cuda, test/test_ops.py::TestCommonCUDA::test_errors_nn_functional_softshrink_cuda, test/test_ops.py::TestCommonCUDA::test_errors_nn_functional_triplet_margin_loss_cuda, test/test_ops.py::TestCommonCUDA::test_errors_normal_in_place_cuda, test/test_ops.py::TestCommonCUDA::test_errors_polar_cuda, test/test_ops.py::TestCommonCUDA::test_errors_signal_windows_blackman_cuda, test/test_ops.py::TestCommonCUDA::test_errors_signal_windows_exponential_cuda, test/test_ops.py::TestCommonCUDA::test_errors_signal_windows_general_hamming_cuda, test/test_ops.py::TestCommonCUDA::test_errors_sparse_randn_like_layout2_cuda, test/test_ops.py::TestCommonCUDA::test_errors_sparse_sum_layout2_cuda, test/test_ops.py::TestCommonCUDA::test_errors_sparse_sum_layout3_cuda, test/test_ops.py::TestCommonCUDA::test_errors_sparse_sum_layout4_cuda, test/test_ops.py::TestCommonCUDA::test_errors_sparse_zeros_like_layout3_cuda, test/test_ops.py::TestCommonCUDA::test_errors_special_chebyshev_polynomial_v_cuda, test/test_ops.py::TestCommonCUDA::test_errors_special_chebyshev_polynomial_w_cuda, test/test_ops.py::TestCommonCUDA::test_errors_special_laguerre_polynomial_l_cuda, test/test_ops.py::TestCommonCUDA::test_errors_special_shifted_chebyshev_polynomial_u_cuda, test/test_ops.py::TestCommonCUDA::test_errors_special_shifted_chebyshev_polynomial_v_cuda, test/test_ops.py::TestCommonCUDA::test_errors_triu_cuda, test/test_ops.py::TestCommonCUDA::test_errors_vstack_cuda, test/test_ops.py::TestCommonCUDA::test_errors_where_cuda, test/test_ops.py::TestCommonCUDA::test_multiple_devices_H_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices___rand___cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices___rmod___cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices___rsub___cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices__segment_reduce_lengths_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices__upsample_bilinear2d_aa_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_acosh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_add_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_addcmul_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_addmm_decomposed_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_addmv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_all_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_angle_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_argmin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_argsort_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_argwhere_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_as_strided_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_as_strided_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_as_strided_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_as_strided_scatter_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_asin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_asinh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_asinh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_atan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_baddbmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_bincount_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_bitwise_left_shift_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_bool_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_broadcast_shapes_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cat_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cat_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cdouble_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cholesky_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_chunk_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_clamp_max_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_column_stack_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_combinations_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_contiguous_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_contiguous_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_copysign_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_copysign_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cos_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cov_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cross_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cumsum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cumulative_trapezoid_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_cumulative_trapezoid_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_diag_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_diagflat_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_diagonal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_diff_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_digamma_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_dist_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_div_floor_rounding_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_div_floor_rounding_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_dot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_dstack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_empty_permuted_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_exponential_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_eye_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fft_fft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fft_ifft_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fft_irfft2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fft_irfft2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fft_irfft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fft_irfftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fft_rfftn_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fill_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_flatten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_flip_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_flip_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fliplr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fliplr_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_float_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_floor_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_floor_divide_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_fmin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_gather_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_geometric_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_hsplit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_hypot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_igammac_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_index_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_index_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_index_put_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_index_reduce_amin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_index_reduce_prod_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_index_select_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_int_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_isclose_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_isfinite_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_item_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_jiterator_binary_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_kron_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_kron_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_kthvalue_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_lcm_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_ldexp_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_le_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_lerp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_lgamma_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_cond_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_cross_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_householder_product_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_inv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_inv_ex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_lu_factor_ex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_lu_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_matrix_rank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_solve_triangular_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_svd_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_svdvals_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linalg_tensorinv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_linspace_tensor_overload_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_log10_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_log10_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_log1p_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_log2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_log_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_log_normal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_logaddexp2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_logaddexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_logical_and_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_logical_and_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_logical_not_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_logical_or_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_logsumexp_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_long_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_mH_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_mT_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_masked_argmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_masked_fill_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_masked_logaddexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_masked_logsumexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_masked_prod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_masked_std_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_masked_sum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_matrix_exp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_max_binary_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_max_pool2d_with_indices_backward_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_max_reduction_with_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_median_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_meshgrid_list_of_tensors_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_min_reduction_with_dim_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_msort_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_mul_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_mul_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nan_to_num_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_narrow_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_ne_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_neg_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_new_empty_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_new_empty_strided_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_new_zeros_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_adaptive_avg_pool2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_adaptive_max_pool1d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_batch_norm_without_cudnn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_binary_cross_entropy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_celu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_channel_shuffle_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_cosine_embedding_loss_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_dropout2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_embedding_bag_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_feature_alpha_dropout_without_train_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_gaussian_nll_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_group_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_hardshrink_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_hardsigmoid_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_hardswish_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_hardtanh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_hardtanh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_huber_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_interpolate_bilinear_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_max_unpool2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_max_unpool3d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_multi_head_attention_forward_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_multi_margin_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_multilabel_margin_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_normalize_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_one_hot_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_pad_circular_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_pad_circular_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_pad_constant_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_pad_reflect_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_pad_replicate_negative_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_pixel_shuffle_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_pixel_unshuffle_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_relu6_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_relu_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_scaled_dot_product_attention_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_softmin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_softshrink_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_triplet_margin_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_triplet_margin_loss_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_nn_functional_triplet_margin_with_distance_loss_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_norm_nuc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_normal_number_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_ones_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_pca_lowrank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_permute_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_polygamma_polygamma_n_0_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_prod_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_rad2deg_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_randint_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_randint_like_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_ravel_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_reciprocal_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_renorm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_repeat_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_resize__cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_resolve_conj_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_scatter_add_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_scatter_reduce_amin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_scatter_reduce_sum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_searchsorted_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_select_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_select_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_signal_windows_blackman_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_signal_windows_cosine_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_signal_windows_general_hamming_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_sinh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_sinh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_bessel_j1_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_chebyshev_polynomial_u_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_chebyshev_polynomial_w_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_hermite_polynomial_h_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_i0e_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_i0e_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_log_ndtr_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_modified_bessel_i1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_modified_bessel_k0_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_modified_bessel_k0_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_modified_bessel_k1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_polygamma_special_polygamma_n_0_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_scaled_modified_bessel_k1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_scaled_modified_bessel_k1_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_shifted_chebyshev_polynomial_u_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_zeta_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_special_zeta_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_split_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_split_list_args_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_sqrt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_std_mean_unbiased_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_sub_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_sum_to_size_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_t_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_t_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_tan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_tensor_split_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_tile_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_topk_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_topk_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_torch_ops_aten__efficient_attention_forward_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_trace_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_tril_indices_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_triu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_triu_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_trunc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_unfold_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_unsafe_chunk_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_var_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_vdot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_view_as_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_view_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_view_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_view_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_vstack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_vstack_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_where_cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_zero__cuda_float32, test/test_ops.py::TestCommonCUDA::test_multiple_devices_zero__cuda_int64, test/test_ops.py::TestCommonCUDA::test_multiple_devices_zeros_like_cuda_float32, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_H_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_T_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values___rdiv___cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values___ror___cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values__unsafe_masked_index_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_acosh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_addr_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_alias_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_amax_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_amin_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_as_strided_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_asinh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_atanh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_bitwise_or_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_bool_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_broadcast_tensors_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_cat_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_chunk_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_conj_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_cummax_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_diag_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_dstack_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_empty_permuted_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_erfc_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_exp2_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_expand_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_fft_ihfft_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_fft_rfft2_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_fft_rfft_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_fill_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_flatten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_flipud_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_fmax_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_index_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_isposinf_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_jiterator_4inputs_with_extra_args_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_le_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_lgamma_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_linalg_diagonal_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_log10_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_log2_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_log_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_logical_not_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_logical_xor_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_mT_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_masked_sum_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_max_reduction_with_dim_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_maximum_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_movedim_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_mul_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_narrow_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_new_empty_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_new_full_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_new_zeros_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_nn_functional_channel_shuffle_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_nn_functional_pad_circular_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_polygamma_polygamma_n_3_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_reciprocal_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_scalar_tensor_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_scatter_reduce_sum_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_select_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_signbit_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_sinh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_special_airy_ai_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_special_bessel_j0_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_special_chebyshev_polynomial_u_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_special_modified_bessel_k0_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_special_scaled_modified_bessel_k1_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_special_shifted_chebyshev_polynomial_t_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_split_list_args_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_split_with_sizes_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_squeeze_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_t_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_take_along_dim_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_take_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_tanh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_tile_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_to_sparse_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_transpose_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_tril_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_unfold_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_unsqueeze_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_vstack_cuda_bool, test/test_ops.py::TestCommonCUDA::test_non_standard_bool_values_zeros_cuda_bool, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples___radd___cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples___rmul___cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples___rpow___cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples__chunk_cat_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples__segment_reduce_offsets_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples__unsafe_masked_index_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_acos_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_acos_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_add_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_addcdiv_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_addcdiv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_addcmul_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_addcmul_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_addmm_decomposed_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_addmv_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_amin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_angle_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_angle_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_arange_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_argmax_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_argmin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_argwhere_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_as_strided_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_as_strided_scatter_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_as_strided_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_atan2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_atan2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_atanh_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_atanh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_atleast_1d_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_baddbmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_bitwise_or_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_bitwise_xor_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_block_diag_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_bool_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_broadcast_shapes_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_broadcast_to_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_broadcast_to_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_bucketize_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_byte_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cartesian_prod_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cat_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cfloat_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cfloat_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_char_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cholesky_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cholesky_solve_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_clamp_min_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_column_stack_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_complex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_conj_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_conj_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_conj_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_constant_pad_nd_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_copysign_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_corrcoef_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cos_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_count_nonzero_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_count_nonzero_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cov_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cross_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cross_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cummin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_cumsum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_diag_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_diagflat_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_diagflat_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_diagflat_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_diagonal_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_diff_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_diff_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_digamma_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_div_floor_rounding_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_div_floor_rounding_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_div_no_rounding_mode_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_double_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_double_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_dsplit_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_dsplit_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_dstack_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_empty_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_empty_permuted_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_erf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_erf_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_erfc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_erfc_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_exp2_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_exp_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_expand_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_expm1_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_fft2_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_fftn_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_hfft2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_hfftn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_ifft2_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_ifft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_ifft_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_ifftn_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_ifftshift_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_ifftshift_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_ihfft_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_irfftn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_irfftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fft_rfftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_flip_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_flip_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fliplr_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_float_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_float_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_floor_divide_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fmax_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fmin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_fmod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_full_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_full_like_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_gather_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_gather_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_gcd_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_geometric_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_geometric_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_geqrf_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_geqrf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_gt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_igamma_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_index_fill_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_index_fill_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_index_put_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_index_put_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_index_reduce_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_index_reduce_mean_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_isfinite_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_isfinite_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_isnan_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_isposinf_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_isreal_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_item_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_jiterator_4inputs_with_extra_args_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_kthvalue_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_le_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_le_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_lerp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_cond_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_cross_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_eig_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_eigvals_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_ldl_factor_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_ldl_factor_ex_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_lu_factor_ex_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_lu_solve_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_matrix_rank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_multi_dot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_pinv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_pinv_hermitian_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_pinv_singular_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_slogdet_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_slogdet_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_solve_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_solve_triangular_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_svd_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_tensorinv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_vander_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linalg_vecdot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_linspace_tensor_overload_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_log10_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_log1p_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_log_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_log_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_log_normal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_log_softmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_log_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_logaddexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_logical_and_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_logical_not_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_logit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_logspace_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_logspace_tensor_overload_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_logsumexp_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_lu_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_lu_unpack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_mH_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_mT_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_masked_amin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_masked_cumprod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_masked_logaddexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_masked_prod_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_masked_softmin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_masked_var_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_max_reduction_no_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_max_reduction_with_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_meshgrid_list_of_tensors_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_meshgrid_variadic_tensors_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_meshgrid_variadic_tensors_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_min_binary_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_mul_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_mv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_mvlgamma_mvlgamma_p_3_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_mvlgamma_mvlgamma_p_3_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_mvlgamma_mvlgamma_p_5_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nansum_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_ne_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_neg_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_new_empty_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nextafter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_binary_cross_entropy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_channel_shuffle_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_conv3d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_cosine_embedding_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_elu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_glu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_hardtanh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_interpolate_bilinear_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_l1_loss_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_l1_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_linear_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_margin_ranking_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_mse_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_multi_margin_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_nll_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_normalize_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_normalize_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_pad_reflect_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_pad_reflect_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_pad_replicate_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_pairwise_distance_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_pairwise_distance_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_pdist_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_pixel_shuffle_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_pixel_unshuffle_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_relu6_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_rms_norm_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_selu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_softshrink_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_threshold_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_threshold_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_triplet_margin_loss_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nn_functional_upsample_nearest_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_nonzero_static_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_norm_nuc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_normal_in_place_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_normal_number_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_ones_like_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_pca_lowrank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_pinverse_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_polygamma_polygamma_n_0_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_polygamma_polygamma_n_1_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_polygamma_polygamma_n_2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_polygamma_polygamma_n_2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_polygamma_polygamma_n_4_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_pow_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_prod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_prod_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_qr_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_qr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_rand_like_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_randint_like_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_ravel_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_reciprocal_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_reciprocal_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_renorm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_resize__cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_resize_as__cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_resolve_neg_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_rot90_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_rot90_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_rsqrt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_rsub_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_scatter_reduce_amin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_scatter_reduce_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_scatter_reduce_sum_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_select_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sigmoid_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sign_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sign_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_signal_windows_exponential_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_signal_windows_hamming_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sin_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sinh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_softmax_with_dtype_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sparse_sampled_addmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_bessel_j0_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_bessel_j1_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_bessel_y0_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_chebyshev_polynomial_w_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_hermite_polynomial_h_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_hermite_polynomial_h_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_i0e_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_i1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_i1_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_legendre_polynomial_p_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_modified_bessel_i0_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_modified_bessel_i0_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_modified_bessel_k1_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_polygamma_special_polygamma_n_0_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_shifted_chebyshev_polynomial_v_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_shifted_chebyshev_polynomial_w_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_xlog1py_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_xlog1py_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_special_zeta_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_split_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_split_list_args_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sqrt_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sqrt_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_squeeze_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_squeeze_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_squeeze_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_std_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_sum_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_svd_lowrank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_t_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_take_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_tan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_tensor_split_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_to_sparse_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_transpose_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_trapz_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_trapz_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_tril_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_tril_indices_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_triu_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_true_divide_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unbind_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unfold_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unique_consecutive_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unique_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unsafe_chunk_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unsafe_chunk_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unsafe_split_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unsafe_split_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unsqueeze_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unsqueeze_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_unsqueeze_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_view_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_view_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_view_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_vsplit_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_where_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_zero__cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_zeros_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_zeros_cuda_int64, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_zeros_like_cuda_float32, test/test_ops.py::TestCommonCUDA::test_noncontiguous_samples_zeros_like_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_addbmm_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_argwhere_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_numpy_ref_argwhere_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_broadcast_tensors_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_broadcast_to_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_numpy_ref_broadcast_to_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_broadcast_to_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_cat_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_clone_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_numpy_ref_clone_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_item_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_numpy_ref_linalg_cross_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_linalg_tensorsolve_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_numpy_ref_linalg_tensorsolve_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_meshgrid_variadic_tensors_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_nn_functional_conv_transpose2d_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_nn_functional_l1_loss_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_numpy_ref_nn_functional_one_hot_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_nn_functional_pairwise_distance_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_nn_functional_pairwise_distance_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_ravel_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_ravel_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_repeat_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_roll_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_numpy_ref_signal_windows_bartlett_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_signal_windows_gaussian_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_signal_windows_hann_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_signal_windows_kaiser_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_signal_windows_nuttall_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_squeeze_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_numpy_ref_squeeze_multiple_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_tile_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_tril_indices_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_unbind_cuda_int64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_view_copy_cuda_float64, test/test_ops.py::TestCommonCUDA::test_numpy_ref_where_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_numpy_ref_where_cuda_float64, test/test_ops.py::TestCommonCUDA::test_out_H_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out___rmatmul___cuda_float32, test/test_ops.py::TestCommonCUDA::test_out___rmul___cuda_float32, test/test_ops.py::TestCommonCUDA::test_out___rpow___cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__batch_norm_with_update_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs__conversions_half_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs__conversions_long_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs__conversions_polar_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_add_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_addcdiv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_addr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_alias_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_as_strided_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_as_strided_partial_views_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_asin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_atleast_1d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_bitwise_not_cuda_int64, test/test_ops.py::TestCommonCUDA::test_out__refs_bitwise_or_cuda_int64, test/test_ops.py::TestCommonCUDA::test_out__refs_clamp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_clamp_min_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_clone_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_conj_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_conj_physical_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_cumsum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_diagonal_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_div_floor_rounding_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_dsplit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_erfc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_exp2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_exp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_fft_fft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_fft_fftshift_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_fft_hfft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_fft_ihfft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_fft_rfft2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_fliplr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_fmod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_ge_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_igammac_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_index_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_isclose_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_isfinite_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_le_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_lerp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_linalg_cross_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_linalg_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_log10_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_log2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_logaddexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_logical_and_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_logical_or_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_logical_xor_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_logspace_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_logspace_tensor_overload_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_logsumexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_masked_fill_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_maximum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_movedim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_ne_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_new_empty_strided_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_new_zeros_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_nn_functional_margin_ranking_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_nn_functional_nll_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_nn_functional_relu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_nn_functional_softshrink_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_nn_functional_threshold_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_normal__in_place_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_positive_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_reciprocal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_renorm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_round_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_rsqrt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_sgn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_sign_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_special_i1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_special_log_ndtr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_special_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_special_multigammaln_mvlgamma_p_1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_special_ndtr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_special_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_sqrt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_std_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_tanh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_tril_indices_cuda_int64, test/test_ops.py::TestCommonCUDA::test_out__refs_true_divide_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_vdot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_view_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__refs_view_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__segment_reduce_lengths_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out__upsample_bilinear2d_aa_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_acos_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_add_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_addbmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_addcdiv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_addmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_all_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_aminmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_argmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_argsort_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_as_strided_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_asin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_bitwise_left_shift_cuda_int64, test/test_ops.py::TestCommonCUDA::test_out_broadcast_shapes_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_bucketize_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_cholesky_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_conj_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_cov_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_cummax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_cumprod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_cumulative_trapezoid_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_diagonal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_diagonal_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_digamma_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_div_floor_rounding_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_equal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_erfinv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_exp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_expand_as_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_exponential_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_fft_fftshift_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_floor_divide_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_frac_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_frexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_full_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_full_like_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_ge_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_gt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_hsplit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_i0_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_igamma_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_imag_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_index_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_index_reduce_amin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_index_select_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_inner_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_integral_dtype__refs_sum_cuda_int16, test/test_ops.py::TestCommonCUDA::test_out_isinf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_ldexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_lerp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_cond_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_eig_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_eigh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_inv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_inv_ex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_ldl_factor_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_lstsq_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_lstsq_grad_oriented_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_norm_subgradients_at_zero_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_solve_triangular_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_linalg_tensorsolve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_log1p_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_log_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_long_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_lt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_lu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_lu_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_lu_unpack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_masked_argmin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_masked_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_masked_softmin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_masked_var_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_matrix_exp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_mm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_mvlgamma_mvlgamma_p_3_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nanmean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nanquantile_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_new_empty_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_new_full_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_new_ones_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_new_zeros_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_adaptive_avg_pool2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_adaptive_max_pool2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_adaptive_max_pool3d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_avg_pool3d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_binary_cross_entropy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_binary_cross_entropy_with_logits_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_conv_transpose1d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_conv_transpose3d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_embedding_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_feature_alpha_dropout_without_train_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_instance_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_interpolate_trilinear_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_linear_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_local_response_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_max_pool2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_max_unpool1d_grad_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_max_unpool2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_max_unpool2d_grad_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_mish_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_mse_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_multilabel_margin_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_multilabel_soft_margin_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_nll_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_normalize_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_pad_constant_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_pad_replicate_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_pad_replicate_negative_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_poisson_nll_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_scaled_dot_product_attention_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_selu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_smooth_l1_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_nn_functional_unfold_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_norm_fro_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_ones_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_ormqr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_permute_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_prod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_put_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_rad2deg_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_randint_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_ravel_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_acosh_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_add_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_addcdiv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_addmm_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_addmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_alias_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_angle_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_asinh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_atan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_baddbmm_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_baddbmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_ceil_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_cross_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_cumprod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_cumsum_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_deg2rad_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_digamma_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_div_floor_rounding_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_dot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_exp2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_expm1_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_fft_fft2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_fft_ifft2_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_fft_ifft2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_fft_ifft_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_fft_irfftn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_fft_irfftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_float_power_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_fmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_fmod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_frac_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_hstack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_hypot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_index_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_index_select_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_kthvalue_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_lerp_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_cross_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_det_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_det_singular_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_eigh_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_eigvalsh_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_inv_ex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_lu_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_matrix_power_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_multi_dot_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_multi_dot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_pinv_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_pinv_hermitian_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_pinv_hermitian_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_qr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_slogdet_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_slogdet_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_solve_ex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_svd_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_svd_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_svdvals_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_tensorsolve_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_tensorsolve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linalg_vector_norm_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_linspace_tensor_overload_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_log10_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_log_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_lu_unpack_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_masked_select_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_max_binary_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_max_reduction_no_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_min_reduction_with_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_minimum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_mm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_mul_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_mvlgamma_mvlgamma_p_3_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_nansum_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_nn_functional_gelu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_nn_functional_normalize_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_normal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_normal_number_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_ormqr_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_polygamma_polygamma_n_2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_polygamma_polygamma_n_3_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_reciprocal_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_renorm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_round_decimals_0_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_round_decimals_3_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_rsqrt_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_scatter_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_scatter_reduce_amax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_sigmoid_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_sigmoid_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_sign_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_sinc_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_sinc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_slice_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_softmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_sort_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_special_i0e_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_special_i1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_special_ndtr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_stack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_take_along_dim_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_take_along_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_take_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_tril_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_requires_grad_error_zeros_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_reshape_as_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_resolve_conj_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_rot90_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_scalar_tensor_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_scatter_reduce_amin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_scatter_reduce_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_scatter_reduce_prod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_select_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_sgn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_signal_windows_bartlett_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_signal_windows_general_hamming_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_signal_windows_nuttall_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_sinh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_slice_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_softmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_special_chebyshev_polynomial_v_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_special_erfcx_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_special_i1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_special_modified_bessel_i0_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_special_modified_bessel_i1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_special_ndtri_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_special_polygamma_special_polygamma_n_0_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_special_xlog1py_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_split_list_args_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_squeeze_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_std_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_std_mean_unbiased_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_t_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_t_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_take_along_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_take_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_tan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_to_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_to_sparse_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_torch_ops_aten__safe_softmax_default_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_trapz_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_triu_indices_cuda_int64, test/test_ops.py::TestCommonCUDA::test_out_unbind_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_unsafe_chunk_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_unsqueeze_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_vdot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_view_as_complex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_view_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_vsplit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_out_warning___getitem___cuda, test/test_ops.py::TestCommonCUDA::test_out_warning___radd___cuda, test/test_ops.py::TestCommonCUDA::test_out_warning___rpow___cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs__conversions_bool_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs__conversions_complex_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs__conversions_half_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_add_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_addcdiv_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_amax_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_amin_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_bitwise_left_shift_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_block_diag_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_bucketize_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_chunk_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_clamp_max_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_conj_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_cosh_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_diagonal_copy_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_diagonal_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_div_floor_rounding_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_div_no_rounding_mode_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_div_trunc_rounding_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_empty_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_empty_strided_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_erfc_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_exp_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_expm1_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_fft_fft2_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_fft_hfftn_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_fft_ifftn_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_fft_ifftshift_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_fft_ihfft2_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_fft_irfftn_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_flipud_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_floor_divide_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_fmax_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_frac_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_gcd_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_index_copy_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_item_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_linalg_vector_norm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_linspace_tensor_overload_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_log1p_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_logaddexp_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_logical_and_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_masked_fill_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_maximum_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_narrow_copy_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_native_layer_norm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_ne_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_new_empty_strided_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_nn_functional_channel_shuffle_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_nn_functional_l1_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_nn_functional_layer_norm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_nn_functional_margin_ranking_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_nn_functional_poisson_nll_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_nn_functional_prelu_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_nn_functional_selu_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_nn_functional_tanhshrink_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_permute_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_pow_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_ravel_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_roll_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_rsub_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_sigmoid_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_sinc_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_sinh_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_special_i0e_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_special_multigammaln_mvlgamma_p_1_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_special_ndtr_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_special_spherical_bessel_j0_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_special_xlog1py_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_squeeze_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_std_mean_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_sum_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_take_along_dim_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_to_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_tril_indices_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_true_divide_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_unflatten_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_unsqueeze_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_vdot_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_view_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__refs_vstack_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__segment_reduce_offsets_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning__upsample_bilinear2d_aa_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_addbmm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_addcdiv_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_addr_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_all_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_amin_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_angle_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_any_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_arange_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_as_strided_scatter_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_atleast_2d_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_bernoulli_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_bincount_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_bitwise_and_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_bitwise_xor_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_block_diag_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_byte_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_cat_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_ceil_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_conj_physical_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_copysign_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_corrcoef_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_cummin_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_cumprod_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_diagonal_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_diagonal_scatter_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_diff_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_empty_like_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_empty_permuted_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_erfinv_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_eye_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_fft_fft2_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_fft_fft_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_fft_hfft2_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_fft_hfftn_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_fft_ifft_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_fft_ihfftn_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_fft_rfft_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_fft_rfftn_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_fill_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_flip_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_float_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_frac_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_frexp_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_full_like_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_gcd_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_geometric_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_gradient_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_heaviside_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_histogram_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_hsplit_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_igammac_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_index_reduce_amin_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_isinf_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_jiterator_binary_return_by_ref_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_linalg_cross_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_linalg_lstsq_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_linalg_lu_factor_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_linalg_matrix_norm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_linalg_multi_dot_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_linalg_slogdet_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_linalg_solve_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_linspace_tensor_overload_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_log10_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_log_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_log_softmax_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_log_softmax_with_dtype_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_logical_or_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_lu_solve_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_mH_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_mT_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_masked_cumsum_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_masked_mean_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_masked_median_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_masked_norm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_masked_normalize_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_mul_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_mv_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nanmean_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_native_dropout_backward_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_native_layer_norm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_new_empty_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_new_zeros_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_adaptive_avg_pool2d_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_adaptive_avg_pool3d_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_adaptive_max_pool3d_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_avg_pool1d_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_avg_pool3d_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_batch_norm_without_cudnn_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_bilinear_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_binary_cross_entropy_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_conv_transpose1d_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_cosine_embedding_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_cross_entropy_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_ctc_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_dropout_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_gaussian_nll_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_gelu_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_group_norm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_hardshrink_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_hardsigmoid_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_huber_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_instance_norm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_interpolate_bicubic_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_interpolate_linear_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_interpolate_trilinear_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_l1_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_layer_norm_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_leaky_relu_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_max_pool1d_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_max_pool2d_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_max_unpool3d_grad_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_multi_margin_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_multilabel_margin_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_multilabel_soft_margin_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_normalize_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_pad_constant_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_pixel_unshuffle_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_softsign_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nn_functional_triplet_margin_with_distance_loss_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_nonzero_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_ormqr_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_polar_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_polygamma_polygamma_n_0_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_qr_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_quantile_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_ravel_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_rot90_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_rsub_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_scatter_reduce_amax_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_scatter_reduce_prod_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_sigmoid_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_signal_windows_hann_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_signal_windows_kaiser_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_sinc_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_softmax_with_dtype_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_sparse_mm_reduce_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_special_airy_ai_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_special_chebyshev_polynomial_t_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_special_scaled_modified_bessel_k0_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_special_shifted_chebyshev_polynomial_t_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_split_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_split_list_args_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_split_with_sizes_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_svd_lowrank_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_t_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_tensor_split_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_to_sparse_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_torch_ops_aten__flash_attention_forward_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_trapz_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_tril_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_unbind_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_vstack_cuda, test/test_ops.py::TestCommonCUDA::test_out_warning_zeros_like_cuda, test/test_ops.py::TestCommonCUDA::test_out_zero__cuda_float32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float___rdiv___cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_acos_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_asin_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_asin_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_asinh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_asinh_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_atan2_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_atan2_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_atan_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_atanh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_atanh_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_copysign_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_copysign_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_cos_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_cos_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_cosh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_deg2rad_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_deg2rad_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_digamma_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_digamma_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_div_no_rounding_mode_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_div_no_rounding_mode_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_div_no_rounding_mode_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_erf_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_erf_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_erf_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_erfc_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_erfc_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_erfc_cuda_int64, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_erfc_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_erfinv_cuda_int64, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_erfinv_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_exp2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_exp2_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_exp_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_expm1_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_float_power_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_i0_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_lgamma_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_lgamma_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log10_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log10_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log10_cuda_int64, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log10_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log1p_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log1p_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log1p_cuda_int64, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log1p_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log2_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_log_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_logit_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_logit_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_logit_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_logit_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_masked_mean_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_masked_mean_cuda_int64, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_masked_mean_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_masked_std_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_masked_std_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_masked_var_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_masked_var_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_masked_var_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_mvlgamma_mvlgamma_p_3_cuda_int64, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_mvlgamma_mvlgamma_p_3_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_mvlgamma_mvlgamma_p_5_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_polygamma_polygamma_n_0_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_polygamma_polygamma_n_0_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_polygamma_polygamma_n_1_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_polygamma_polygamma_n_3_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_polygamma_polygamma_n_3_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_rad2deg_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_rad2deg_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_reciprocal_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_rsqrt_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_rsqrt_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_rsqrt_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_sigmoid_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_sigmoid_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_sinc_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_sinc_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_sinc_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_sinh_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_chebyshev_polynomial_t_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_chebyshev_polynomial_t_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_chebyshev_polynomial_v_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_chebyshev_polynomial_v_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_chebyshev_polynomial_v_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_hermite_polynomial_h_cuda_int64, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_laguerre_polynomial_l_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_legendre_polynomial_p_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_shifted_chebyshev_polynomial_t_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_shifted_chebyshev_polynomial_t_cuda_int64, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_shifted_chebyshev_polynomial_u_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_shifted_chebyshev_polynomial_u_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_shifted_chebyshev_polynomial_v_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_shifted_chebyshev_polynomial_v_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_shifted_chebyshev_polynomial_w_cuda_int8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_special_xlog1py_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_sqrt_cuda_int16, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_tan_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_tanh_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_true_divide_cuda_bool, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_true_divide_cuda_int32, test/test_ops.py::TestCommonCUDA::test_promotes_int_to_float_true_divide_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_T_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_bfloat16_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_bfloat16_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_bool_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_bool_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_bool_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_bool_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_bool_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_bool_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_byte_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_byte_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cdouble_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cdouble_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cdouble_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cdouble_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cdouble_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cfloat_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cfloat_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cfloat_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cfloat_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_cfloat_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_chalf_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_chalf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_chalf_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_chalf_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_char_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_char_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_double_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_double_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_double_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_double_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_double_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_double_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_float_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_float_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_float_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_half_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_int_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_int_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_int_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_int_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_long_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_long_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_long_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_long_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_short_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_short_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs__conversions_short_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_abs_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_abs_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_abs_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_acos_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_acos_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_acosh_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_acosh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_acosh_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_add_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_add_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_add_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_addcdiv_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_addcmul_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_addcmul_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_addcmul_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_addr_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_addr_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_addr_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_alias_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_alias_copy_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_alias_copy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_alias_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_alias_copy_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_all_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_all_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_all_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_amax_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_amax_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_amax_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_amin_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_amin_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_any_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_any_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_any_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_any_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_arange_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_arange_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_copy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_partial_views_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_partial_views_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_partial_views_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_partial_views_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_scatter_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_as_strided_scatter_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_asin_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_asin_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_asin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_asinh_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_asinh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_asinh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atan2_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atan2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atan_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atan_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atan_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atanh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atanh_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atanh_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atanh_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atanh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atanh_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_1d_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_1d_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_2d_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_2d_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_2d_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_2d_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_3d_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_3d_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_atleast_3d_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_and_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_left_shift_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_left_shift_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_left_shift_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_not_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_not_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_or_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_or_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_right_shift_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_xor_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bitwise_xor_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_block_diag_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_block_diag_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_broadcast_tensors_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_broadcast_tensors_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_broadcast_tensors_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_broadcast_to_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_broadcast_to_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_broadcast_to_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_bucketize_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_cat_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_cat_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_cauchy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_cauchy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ceil_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_chunk_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_chunk_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_clamp_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_clamp_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_clamp_max_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_clamp_max_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_clamp_min_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_clamp_min_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_clone_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_clone_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_column_stack_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_column_stack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_column_stack_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_column_stack_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_conj_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_conj_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_conj_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_conj_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_conj_physical_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_conj_physical_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_constant_pad_nd_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_constant_pad_nd_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_constant_pad_nd_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_constant_pad_nd_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_constant_pad_nd_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_contiguous_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_contiguous_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_contiguous_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_contiguous_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_contiguous_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_copysign_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_copysign_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_cos_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_cos_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_cos_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_count_nonzero_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_count_nonzero_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_count_nonzero_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_cumprod_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_cumsum_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_deg2rad_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_deg2rad_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_deg2rad_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diag_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diag_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diag_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diag_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diag_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diag_embed_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diag_embed_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_copy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_scatter_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_diagonal_scatter_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_digamma_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_digamma_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_digamma_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_floor_rounding_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_floor_rounding_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_floor_rounding_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_no_rounding_mode_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_no_rounding_mode_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_no_rounding_mode_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_no_rounding_mode_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_trunc_rounding_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_trunc_rounding_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_div_trunc_rounding_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_dot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_dsplit_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_dsplit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_dstack_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_dstack_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_empty_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_empty_like_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_empty_like_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_empty_like_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_empty_like_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_empty_like_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_empty_strided_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_empty_strided_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_eq_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_eq_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_eq_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_equal_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_equal_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_equal_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_erf_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_erf_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_erf_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_erf_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_erfc_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_erfinv_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_erfinv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_erfinv_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_exp2_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_exp_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_exp_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expand_as_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expand_copy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expand_copy_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expand_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expand_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expand_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expand_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expm1_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expm1_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_expm1_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_exponential_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_eye_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fft2_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fft_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fft_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fftn_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fftn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fftn_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fftn_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fftn_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fftshift_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_fftshift_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_hfft2_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_hfft2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_hfft2_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_hfft_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_hfft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_hfft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_hfft_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_hfftn_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_hfftn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ifft2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ifft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ifft_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ifftn_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ifftshift_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ifftshift_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ifftshift_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ihfft2_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ihfft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ihfft_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_ihfftn_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_irfft2_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_irfft2_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_irfft_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_irfft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_irfft_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_irfftn_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_irfftn_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_irfftn_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_irfftn_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_rfft2_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_rfft_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_rfft_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_rfft_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_rfft_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fft_rfftn_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fill_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flatten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flatten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flatten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flatten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flip_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flip_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flip_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flip_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fliplr_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fliplr_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flipud_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flipud_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flipud_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flipud_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flipud_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_flipud_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_float_power_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_float_power_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_float_power_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_float_power_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_float_power_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_floor_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_floor_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_floor_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_floor_divide_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_floor_divide_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_floor_divide_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fmax_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fmax_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fmin_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_fmin_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_frac_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ge_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ge_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ge_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_geometric_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_geometric_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_geometric_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_geometric_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_gt_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_gt_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_gt_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_gt_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_heaviside_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_heaviside_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_heaviside_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_heaviside_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hsplit_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hsplit_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hsplit_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hsplit_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hstack_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hstack_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hstack_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hstack_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hstack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hstack_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hstack_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hstack_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hstack_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hypot_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_hypot_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_i0_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_i0_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_i0_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_igamma_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_add_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_add_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_add_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_copy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_fill_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_fill_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_fill_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_select_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_select_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_index_select_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isclose_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isclose_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isclose_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isfinite_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isfinite_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isfinite_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isfinite_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isinf_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isinf_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isinf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isinf_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isinf_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isinf_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isnan_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isnan_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isnan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isnan_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isneginf_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isneginf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isneginf_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isneginf_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isposinf_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isposinf_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isposinf_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isreal_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isreal_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isreal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isreal_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isreal_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isreal_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_isreal_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_item_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_item_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_item_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_item_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_item_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_item_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_lcm_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_le_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_le_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_lerp_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_lgamma_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_lgamma_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_cross_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_cross_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_cross_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_cross_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_diagonal_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_diagonal_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_matrix_norm_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_svd_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_svdvals_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_svdvals_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_vecdot_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_vector_norm_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linalg_vector_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linspace_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linspace_tensor_overload_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linspace_tensor_overload_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_linspace_tensor_overload_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log10_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log10_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log1p_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log2_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log2_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log_normal_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log_softmax_with_dtype_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log_softmax_with_dtype_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_log_softmax_with_dtype_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logaddexp2_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logaddexp2_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_and_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_and_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_and_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_and_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_not_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_not_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_not_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_or_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_xor_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_xor_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logical_xor_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logspace_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logspace_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logspace_tensor_overload_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logspace_tensor_overload_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logspace_tensor_overload_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logspace_tensor_overload_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logspace_tensor_overload_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logspace_tensor_overload_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logsumexp_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_logsumexp_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_lt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_lt_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_lt_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_lt_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_masked_fill_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_masked_fill_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_masked_fill_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_maximum_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_meshgrid_list_of_tensors_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_meshgrid_list_of_tensors_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_meshgrid_variadic_tensors_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_meshgrid_variadic_tensors_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_meshgrid_variadic_tensors_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_meshgrid_variadic_tensors_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_meshgrid_variadic_tensors_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_meshgrid_variadic_tensors_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_minimum_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_minimum_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_minimum_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_minimum_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_movedim_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_movedim_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_mul_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_mul_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nan_to_num_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nan_to_num_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nan_to_num_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nan_to_num_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_narrow_copy_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_narrow_copy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_narrow_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_narrow_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_narrow_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_native_layer_norm_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_native_layer_norm_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ne_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ne_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ne_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ne_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ne_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_neg_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_neg_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_neg_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_empty_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_empty_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_empty_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_empty_strided_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_empty_strided_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_full_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_full_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_ones_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_ones_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_zeros_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_new_zeros_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nextafter_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_alpha_dropout_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_celu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_celu_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_channel_shuffle_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_channel_shuffle_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_dropout_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_gelu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_glu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_group_norm_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_group_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_hardtanh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_hardtanh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_huber_loss_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_huber_loss_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_l1_loss_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_l1_loss_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_l1_loss_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_leaky_relu_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_leaky_relu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_log_softmax_with_dtype_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_log_softmax_with_dtype_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_margin_ranking_loss_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_margin_ranking_loss_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_margin_ranking_loss_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_margin_ranking_loss_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_mish_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_mse_loss_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_mse_loss_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_nll_loss_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pairwise_distance_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pairwise_distance_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pairwise_distance_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pairwise_distance_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_shuffle_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_shuffle_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_shuffle_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_shuffle_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_unshuffle_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_unshuffle_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_unshuffle_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_unshuffle_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_unshuffle_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_pixel_unshuffle_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_poisson_nll_loss_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_poisson_nll_loss_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_prelu_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_relu6_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_relu6_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_relu_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_relu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_relu_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_relu_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_selu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_smooth_l1_loss_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_softmax_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_softmax_with_dtype_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_softmin_with_dtype_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_softmin_with_dtype_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_softplus_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_tanhshrink_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_threshold_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_nn_functional_threshold_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_norm_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_normal_number_mean_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ones_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_permute_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_permute_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_positive_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_positive_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_positive_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_positive_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_positive_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_pow_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_pow_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_pow_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_pow_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_pow_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_pow_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_prod_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_prod_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rad2deg_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rad2deg_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rad2deg_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ravel_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ravel_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ravel_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ravel_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_ravel_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_real_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_real_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_real_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_real_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reciprocal_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reciprocal_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_renorm_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_renorm_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_repeat_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_repeat_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_repeat_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_repeat_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reshape_as_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reshape_as_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reshape_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reshape_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reshape_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reshape_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reshape_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_reshape_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_roll_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_roll_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_roll_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_roll_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_roll_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_roll_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rot90_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rot90_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rot90_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rot90_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_round_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rsqrt_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rsqrt_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rsqrt_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_rsqrt_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_select_scatter_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sgn_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sgn_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sgn_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sgn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sgn_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sigmoid_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sigmoid_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sigmoid_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sigmoid_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sigmoid_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sign_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sign_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sign_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_signbit_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_signbit_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_signbit_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_signbit_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sin_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sin_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sin_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sin_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sinc_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sinc_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sinc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sinc_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sinh_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sinh_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sinh_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_softmax_with_dtype_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_bessel_j0_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_bessel_j1_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_entr_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_entr_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_erfcx_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_erfcx_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_i0e_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_i0e_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_i0e_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_i0e_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_i1_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_i1e_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_log_ndtr_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_log_softmax_with_dtype_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_log_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_log_softmax_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_log_softmax_with_dtype_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_logit_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_logit_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_multigammaln_mvlgamma_p_1_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_multigammaln_mvlgamma_p_1_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_multigammaln_mvlgamma_p_3_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_multigammaln_mvlgamma_p_5_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_ndtr_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_ndtri_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_softmax_with_dtype_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_softmax_with_dtype_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_softmax_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_spherical_bessel_j0_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_spherical_bessel_j0_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_xlog1py_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_xlog1py_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_zeta_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_special_zeta_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sqrt_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sqrt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sqrt_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sqrt_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_square_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_square_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_square_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_squeeze_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_squeeze_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_squeeze_multiple_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_squeeze_multiple_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_stack_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_std_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_std_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_std_mean_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_std_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sub_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sum_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sum_to_size_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sum_to_size_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_sum_to_size_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_t_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_t_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_t_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_t_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_t_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_take_along_dim_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tan_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tan_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tan_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tanh_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tanh_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tanh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tanh_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tanh_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tensor_split_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tensor_split_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tensor_split_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_to_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_to_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_trace_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_trace_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_trace_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_trace_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_trace_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_transpose_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_transpose_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_transpose_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tril_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tril_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tril_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_tril_indices_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_triu_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_triu_indices_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_true_divide_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_trunc_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_trunc_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unbind_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unbind_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unbind_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unbind_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unflatten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unflatten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unflatten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unfold_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unfold_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unfold_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unfold_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unfold_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unfold_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unfold_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unsqueeze_copy_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unsqueeze_copy_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unsqueeze_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unsqueeze_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unsqueeze_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unsqueeze_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_unsqueeze_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_var_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_var_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_var_mean_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_view_as_complex_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_view_copy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_view_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_view_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_view_copy_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_view_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_view_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_vsplit_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_vsplit_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_vsplit_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_vstack_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_vstack_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_where_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_where_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_xlogy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_xlogy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_xlogy_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_xlogy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_xlogy_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_zeros_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_zeros_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_zeros_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_zeros_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref__refs_zeros_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs__conversions_complex_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs__conversions_polar_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_amin_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_as_strided_scatter_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_bitwise_left_shift_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_diag_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_exponential_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_eye_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_fft_ihfft2_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_fft_ihfft_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_float_power_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_heaviside_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_hsplit_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_hstack_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_le_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_linspace_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_maximum_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_narrow_copy_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_narrow_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_native_layer_norm_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_nn_functional_softshrink_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_nn_functional_triplet_margin_loss_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_normal__in_place_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_pow_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_renorm_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_roll_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_rot90_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_rsub_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_trace_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_tril_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_vdot_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_errors__refs_view_as_cuda, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_T_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_T_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_bfloat16_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_bfloat16_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_bfloat16_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_bfloat16_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_bfloat16_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_bool_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_bool_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_bool_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_bool_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_byte_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_byte_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_byte_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_byte_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_byte_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_cdouble_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_cdouble_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_cdouble_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_cfloat_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_cfloat_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_chalf_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_chalf_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_chalf_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_chalf_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_chalf_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_char_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_char_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_char_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_char_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_double_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_double_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_double_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_float_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_float_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_half_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_half_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_half_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_half_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_int_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_int_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_int_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_int_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_long_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_short_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_short_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs__conversions_short_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_abs_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_abs_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_abs_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acos_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acos_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acos_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acos_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acos_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acos_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acos_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acos_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acosh_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acosh_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_acosh_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_add_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_add_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_add_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_addcdiv_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_addcmul_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_addcmul_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_addcmul_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_addr_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_addr_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_alias_copy_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_alias_copy_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_alias_copy_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_all_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_all_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_all_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_all_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_all_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_all_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_all_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_amax_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_amax_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_amin_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_amin_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_amin_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_any_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_any_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_arange_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_arange_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_arange_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_copy_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_copy_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_partial_views_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_partial_views_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_partial_views_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_scatter_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_scatter_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_scatter_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_as_strided_scatter_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asin_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asin_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asin_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asin_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asin_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asin_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asinh_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asinh_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asinh_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asinh_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asinh_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asinh_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_asinh_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atan2_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atan2_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atan2_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atan2_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atan_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atan_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atanh_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atanh_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atanh_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atleast_1d_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atleast_1d_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atleast_1d_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atleast_2d_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atleast_2d_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_atleast_2d_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_bitwise_and_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_bitwise_left_shift_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_bitwise_or_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_bitwise_right_shift_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_bitwise_xor_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_bitwise_xor_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_bitwise_xor_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_block_diag_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_block_diag_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_block_diag_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_broadcast_shapes_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_broadcast_tensors_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_broadcast_tensors_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_broadcast_to_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_broadcast_to_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_broadcast_to_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_broadcast_to_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_broadcast_to_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cat_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cat_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cat_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cauchy_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_ceil_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_chunk_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clamp_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clamp_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clamp_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clamp_max_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clamp_max_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clamp_min_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clamp_min_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clamp_min_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clone_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clone_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_clone_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_column_stack_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_column_stack_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_column_stack_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_column_stack_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_column_stack_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_conj_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_conj_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_conj_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_conj_physical_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_conj_physical_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_conj_physical_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_constant_pad_nd_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_constant_pad_nd_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_contiguous_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_copysign_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_copysign_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_copysign_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cos_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cos_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cosh_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cosh_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cosh_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cosh_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_count_nonzero_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_count_nonzero_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_count_nonzero_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_count_nonzero_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cumprod_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cumsum_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cumsum_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_cumsum_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_deg2rad_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diag_embed_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diag_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_copy_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_copy_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_scatter_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_diagonal_scatter_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_digamma_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_digamma_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_digamma_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_div_floor_rounding_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_div_no_rounding_mode_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_div_no_rounding_mode_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_div_no_rounding_mode_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_div_trunc_rounding_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_div_trunc_rounding_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_dot_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_dot_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_dot_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_dsplit_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_dsplit_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_dstack_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_dstack_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_dstack_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_empty_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_empty_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_empty_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_empty_like_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_empty_like_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_empty_like_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_empty_strided_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_eq_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_eq_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_eq_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_equal_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_equal_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_equal_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_erf_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_erf_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_erfc_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_erfinv_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_exp2_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_exp2_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_exp_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_exp_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_exp_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_exp_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expand_as_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expand_as_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expand_as_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expand_copy_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expand_copy_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expand_copy_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expand_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expand_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expm1_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expm1_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_expm1_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_exponential_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_eye_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_fft2_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_fft_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_fft_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_fft_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_fft_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_fftn_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_fftshift_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_fftshift_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_hfft2_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_hfft2_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_hfft_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_hfft_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_hfft_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_hfft_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_hfftn_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_hfftn_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_hfftn_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifft2_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifft2_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifft2_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifft2_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifft_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifft_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifft_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifft_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifftn_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifftn_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ifftshift_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ihfft2_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ihfft2_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ihfft_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_ihfftn_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_irfft_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_irfft_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_irfft_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_irfftn_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_rfft_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_rfft_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_rfftn_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fft_rfftn_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fill_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_flatten_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_flatten_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_flatten_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_flip_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_flip_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fliplr_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fliplr_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_flipud_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_flipud_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_float_power_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_float_power_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_floor_divide_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_floor_divide_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_floor_divide_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_floor_divide_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_floor_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_floor_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_floor_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_floor_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fmax_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fmin_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fmin_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fmin_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fmod_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_fmod_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_frac_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_frexp_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_gcd_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_ge_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_geometric_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_geometric_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_gt_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_gt_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_gt_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_heaviside_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_heaviside_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_heaviside_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_heaviside_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_heaviside_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_heaviside_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hsplit_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hsplit_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hsplit_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hstack_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hstack_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hstack_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hstack_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hstack_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hstack_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_hypot_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_i0_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_i0_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_igammac_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_igammac_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_add_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_add_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_add_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_add_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_copy_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_copy_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_copy_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_fill_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_fill_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_fill_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_select_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_select_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_select_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_select_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_select_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_index_select_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isclose_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isclose_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isclose_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isclose_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isclose_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isfinite_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isfinite_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isfinite_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isfinite_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isinf_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isinf_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isinf_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isinf_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isnan_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isnan_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isnan_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isposinf_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isposinf_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isposinf_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isposinf_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isposinf_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isreal_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_isreal_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_istft_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_item_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_item_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_item_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_item_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_item_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_lcm_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_le_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_le_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_lgamma_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_lgamma_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_lgamma_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_lgamma_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_cross_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_diagonal_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_diagonal_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_diagonal_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_diagonal_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_matrix_norm_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_matrix_norm_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_matrix_norm_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_norm_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_norm_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_svd_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_svdvals_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_vecdot_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_vector_norm_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_vector_norm_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linalg_vector_norm_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linspace_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linspace_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linspace_tensor_overload_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linspace_tensor_overload_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linspace_tensor_overload_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_linspace_tensor_overload_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log10_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log10_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log10_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log10_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log10_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log1p_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log1p_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log2_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log2_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log2_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log2_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log_normal_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log_normal_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log_normal_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log_softmax_with_dtype_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log_softmax_with_dtype_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log_softmax_with_dtype_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_log_softmax_with_dtype_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logaddexp2_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logaddexp2_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logaddexp2_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logaddexp_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logaddexp_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logical_and_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logical_and_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logical_not_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logical_or_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logical_or_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logspace_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logspace_tensor_overload_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logspace_tensor_overload_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logspace_tensor_overload_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logsumexp_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_logsumexp_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_lt_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_lt_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_lt_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_lt_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_masked_fill_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_masked_fill_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_masked_fill_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_maximum_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_maximum_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_maximum_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_maximum_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_maximum_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_meshgrid_list_of_tensors_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_meshgrid_list_of_tensors_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_meshgrid_variadic_tensors_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_meshgrid_variadic_tensors_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_meshgrid_variadic_tensors_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_meshgrid_variadic_tensors_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_minimum_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_movedim_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_movedim_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_movedim_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_movedim_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_movedim_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_movedim_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_mul_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_mul_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_mul_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_mul_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nan_to_num_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nan_to_num_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_narrow_copy_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_narrow_copy_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_narrow_copy_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_narrow_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_narrow_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_narrow_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_narrow_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_ne_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_neg_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_neg_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_neg_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_empty_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_empty_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_empty_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_empty_strided_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_empty_strided_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_empty_strided_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_empty_strided_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_empty_strided_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_empty_strided_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_full_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_full_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_full_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_full_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_full_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_ones_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_ones_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_ones_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_ones_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_new_zeros_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nextafter_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_alpha_dropout_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_alpha_dropout_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_alpha_dropout_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_celu_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_channel_shuffle_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_channel_shuffle_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_channel_shuffle_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_channel_shuffle_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_dropout_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_gelu_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_group_norm_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_hardtanh_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_hardtanh_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_hardtanh_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_hinge_embedding_loss_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_huber_loss_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_l1_loss_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_leaky_relu_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_log_softmax_with_dtype_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_log_softmax_with_dtype_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_log_softmax_with_dtype_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_margin_ranking_loss_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_margin_ranking_loss_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_margin_ranking_loss_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_mish_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_mse_loss_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_nll_loss_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_nll_loss_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_pixel_shuffle_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_pixel_shuffle_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_pixel_shuffle_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_pixel_unshuffle_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_pixel_unshuffle_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_pixel_unshuffle_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_pixel_unshuffle_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_poisson_nll_loss_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_poisson_nll_loss_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_prelu_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_prelu_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_relu6_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_relu6_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_relu6_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_relu_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_selu_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_selu_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_smooth_l1_loss_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_softmax_with_dtype_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_softmax_with_dtype_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_softmin_with_dtype_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_tanhshrink_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_tanhshrink_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_tanhshrink_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_tanhshrink_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_threshold_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_triplet_margin_loss_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_triplet_margin_loss_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_triplet_margin_loss_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_triplet_margin_loss_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_nn_functional_triplet_margin_loss_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_norm_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_norm_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_normal__in_place_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_normal_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_normal_number_mean_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_ones_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_ones_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_permute_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_permute_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_permute_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_permute_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_permute_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_permute_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_permute_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_positive_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_positive_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_positive_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_pow_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_pow_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_prod_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_prod_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_prod_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_rad2deg_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_rad2deg_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_randn_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_ravel_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_real_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_real_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_real_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reciprocal_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reciprocal_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reciprocal_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_remainder_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_remainder_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_renorm_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_renorm_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_repeat_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_as_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_as_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_as_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_as_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_reshape_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_roll_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_roll_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_roll_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_rot90_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_rot90_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_rot90_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_round_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_round_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_round_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_rsqrt_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_rsqrt_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_rsub_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_rsub_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_select_scatter_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_select_scatter_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sgn_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sgn_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sgn_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sigmoid_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sigmoid_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sigmoid_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sigmoid_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sign_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sign_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sign_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_signbit_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_signbit_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_signbit_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sin_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sin_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sin_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sinc_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sinc_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sinh_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sinh_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sinh_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sinh_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_softmax_with_dtype_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_softmax_with_dtype_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_softmax_with_dtype_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_bessel_j0_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_bessel_j0_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_bessel_j0_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_bessel_j1_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_bessel_j1_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_entr_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_entr_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_entr_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_entr_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_entr_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_erfcx_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_erfcx_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_erfcx_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_i0e_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_i0e_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_i0e_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_i1_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_i1_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_i1e_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_i1e_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_i1e_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_i1e_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_log_ndtr_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_log_ndtr_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_log_softmax_with_dtype_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_log_softmax_with_dtype_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_log_softmax_with_dtype_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_log_softmax_with_dtype_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_log_softmax_with_dtype_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_logit_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_logit_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_logit_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_multigammaln_mvlgamma_p_1_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_multigammaln_mvlgamma_p_1_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_multigammaln_mvlgamma_p_1_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_multigammaln_mvlgamma_p_1_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_multigammaln_mvlgamma_p_3_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_ndtr_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_ndtr_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_ndtr_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_ndtr_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_ndtri_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_softmax_with_dtype_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_softmax_with_dtype_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_softmax_with_dtype_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_softmax_with_dtype_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_softmax_with_dtype_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_spherical_bessel_j0_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_zeta_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_zeta_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_zeta_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_special_zeta_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sqrt_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sqrt_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sqrt_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sqrt_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sqrt_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sqrt_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sqrt_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sqrt_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_square_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_square_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_squeeze_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_squeeze_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_squeeze_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_squeeze_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_squeeze_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_squeeze_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_stack_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_stack_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_std_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_std_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_std_mean_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_std_mean_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_stft_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_stft_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sub_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sum_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sum_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sum_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sum_to_size_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sum_to_size_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_sum_to_size_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_t_copy_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_t_copy_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_t_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_t_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_t_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_t_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_take_along_dim_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_take_along_dim_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_take_along_dim_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_take_along_dim_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tan_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tan_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tan_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tanh_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tanh_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tanh_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tanh_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tensor_split_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_to_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_to_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_to_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_trace_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_trace_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_trace_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_transpose_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_transpose_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_transpose_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tril_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tril_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_tril_indices_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_triu_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_triu_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_triu_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_triu_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_triu_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_triu_indices_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_true_divide_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_true_divide_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_trunc_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_trunc_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unbind_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unbind_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unflatten_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unflatten_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unfold_copy_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unfold_copy_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unfold_copy_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unfold_copy_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unfold_copy_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unfold_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unfold_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unsqueeze_copy_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unsqueeze_copy_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unsqueeze_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unsqueeze_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unsqueeze_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unsqueeze_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unsqueeze_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_unsqueeze_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_var_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_var_mean_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_vdot_executor_aten_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_vdot_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_vdot_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_view_as_executor_aten_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_view_as_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_view_as_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_view_as_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_view_as_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_view_executor_aten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_view_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_view_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_view_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_vsplit_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_vsplit_executor_aten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_vstack_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_vstack_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_vstack_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_where_executor_aten_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_where_executor_aten_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_where_executor_aten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_where_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_xlogy_executor_aten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_xlogy_executor_aten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_xlogy_executor_aten_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_xlogy_executor_aten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_zeros_executor_aten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_executor__refs_zeros_executor_aten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_T_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_T_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_T_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_bfloat16_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_bfloat16_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_bfloat16_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_bool_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_bool_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_byte_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_byte_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_byte_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_byte_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_cdouble_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_cdouble_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_cfloat_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_cfloat_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_cfloat_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_chalf_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_chalf_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_chalf_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_chalf_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_chalf_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_char_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_char_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_char_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_char_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_char_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_char_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_complex_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_double_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_double_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_float_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_half_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_half_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_half_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_int_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_long_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_long_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_long_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_long_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_long_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_long_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_short_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs__conversions_short_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_abs_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_abs_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_abs_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_abs_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_acos_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_acos_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_acos_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_acosh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_acosh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_acosh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_acosh_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_add_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_add_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_add_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_addcdiv_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_addcmul_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_addcmul_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_addr_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_alias_copy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_alias_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_alias_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_alias_copy_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_all_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_all_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_all_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_all_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_all_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_allclose_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_allclose_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_amax_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_amin_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_any_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_any_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_any_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_any_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_arange_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_copy_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_partial_views_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_partial_views_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_scatter_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_scatter_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_as_strided_scatter_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_asin_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_asinh_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_asinh_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atan_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atan_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atan_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atanh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atanh_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atanh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atanh_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atanh_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atleast_1d_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atleast_1d_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atleast_2d_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atleast_3d_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_atleast_3d_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_bitwise_or_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_bitwise_or_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_bitwise_right_shift_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_bitwise_right_shift_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_bitwise_xor_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_block_diag_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_block_diag_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_block_diag_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_broadcast_tensors_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_broadcast_tensors_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_broadcast_tensors_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_broadcast_to_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_broadcast_to_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_broadcast_to_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_bucketize_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_bucketize_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cat_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cat_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cat_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cat_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cat_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cauchy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ceil_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ceil_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ceil_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_chunk_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_chunk_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_chunk_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_chunk_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_chunk_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_clamp_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_clamp_max_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_clamp_max_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_clamp_max_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_clamp_min_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_clamp_min_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_clamp_min_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_clamp_min_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_clone_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_column_stack_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_column_stack_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_conj_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_conj_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_conj_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_conj_physical_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_conj_physical_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_conj_physical_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_conj_physical_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_conj_physical_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_conj_physical_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_constant_pad_nd_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_constant_pad_nd_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_constant_pad_nd_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_constant_pad_nd_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_constant_pad_nd_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_contiguous_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_contiguous_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_contiguous_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_contiguous_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_copysign_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cos_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cos_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cos_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cos_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cosh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cosh_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cosh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cosh_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cosh_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cosh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_count_nonzero_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_count_nonzero_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_count_nonzero_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_count_nonzero_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_count_nonzero_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cumprod_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cumprod_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cumsum_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cumsum_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_cumsum_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_deg2rad_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_deg2rad_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_deg2rad_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diag_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diag_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diag_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diag_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diag_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diag_embed_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diag_embed_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_copy_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_copy_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_scatter_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_scatter_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_scatter_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_diagonal_scatter_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_digamma_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_digamma_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_div_no_rounding_mode_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_div_trunc_rounding_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_div_trunc_rounding_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_div_trunc_rounding_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dot_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dot_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dot_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dsplit_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dsplit_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dsplit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dsplit_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dstack_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dstack_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dstack_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dstack_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_dstack_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_empty_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_empty_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_empty_like_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_empty_like_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_empty_like_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_empty_like_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_empty_like_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_empty_like_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_empty_strided_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_eq_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_eq_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_eq_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_eq_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_equal_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_equal_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_equal_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_equal_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_equal_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_erf_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_erf_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_erfc_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_erfc_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_erfc_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_erfc_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_erfc_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_erfinv_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_exp2_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_exp2_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_exp2_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_exp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_exp_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_expand_as_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_expand_as_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_expand_copy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_expand_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_expm1_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_expm1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_expm1_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_exponential_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_exponential_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_exponential_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_eye_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_eye_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_eye_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fft2_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fft_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fft_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fftn_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fftn_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fftn_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fftshift_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fftshift_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fftshift_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_fftshift_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_hfft2_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_hfft_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_hfft_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_hfft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_hfft_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_hfftn_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_hfftn_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ifftn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ifftn_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ifftshift_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ifftshift_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ifftshift_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ifftshift_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ihfft2_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ihfftn_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ihfftn_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_ihfftn_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_irfft2_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_irfft2_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_irfft_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_irfft_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_irfftn_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_irfftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_irfftn_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_rfft2_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_rfft2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_rfft_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_rfft_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_rfftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_rfftn_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fft_rfftn_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fill_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fill_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fill_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flatten_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flatten_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flip_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flip_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flip_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flip_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flip_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fliplr_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fliplr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flipud_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flipud_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flipud_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_flipud_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_float_power_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_float_power_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_float_power_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_floor_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_floor_divide_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_floor_divide_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_floor_divide_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fmax_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fmin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fmin_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fmin_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fmod_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_fmod_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_frac_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_frexp_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_gcd_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ge_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_geometric_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_geometric_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_geometric_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_gt_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_gt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_gt_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_gt_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_heaviside_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_heaviside_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_hsplit_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_hsplit_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_hsplit_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_hsplit_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_hsplit_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_hsplit_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_hstack_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_hstack_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_hypot_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_i0_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_i0_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_i0_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_igamma_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_igammac_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_igammac_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_imag_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_add_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_add_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_add_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_copy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_copy_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_fill_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_fill_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_fill_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_select_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_select_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_index_select_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isclose_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isclose_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isclose_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isclose_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isfinite_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isfinite_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isfinite_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isinf_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isinf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isinf_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isnan_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isnan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isneginf_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isneginf_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isneginf_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isposinf_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isposinf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isposinf_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isposinf_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isreal_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isreal_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isreal_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isreal_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isreal_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_isreal_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_istft_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_item_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_lcm_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_le_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_le_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_le_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_lerp_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_lerp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_lgamma_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_lgamma_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_lgamma_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_cross_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_cross_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_diagonal_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_diagonal_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_diagonal_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_diagonal_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_norm_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_svd_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linalg_vector_norm_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linspace_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linspace_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linspace_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linspace_tensor_overload_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_linspace_tensor_overload_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log10_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log10_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log10_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log1p_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log1p_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log1p_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log1p_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log1p_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log2_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log_normal_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log_normal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log_softmax_with_dtype_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_log_softmax_with_dtype_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logaddexp2_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_and_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_and_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_and_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_not_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_not_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_not_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_not_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_or_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_or_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logical_or_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logspace_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logspace_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logspace_tensor_overload_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logsumexp_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logsumexp_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logsumexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logsumexp_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_logsumexp_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_lt_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_lt_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_lt_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_masked_fill_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_masked_fill_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_masked_fill_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_masked_fill_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_maximum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_maximum_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_mean_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_mean_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_mean_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_meshgrid_variadic_tensors_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_meshgrid_variadic_tensors_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_meshgrid_variadic_tensors_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_minimum_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_minimum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_minimum_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_movedim_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_mul_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_mul_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nan_to_num_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nan_to_num_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_narrow_copy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_narrow_copy_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_narrow_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_narrow_copy_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_narrow_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_narrow_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ne_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ne_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ne_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ne_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_neg_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_neg_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_neg_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_neg_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_empty_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_empty_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_empty_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_empty_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_empty_strided_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_empty_strided_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_full_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_full_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_full_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_ones_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_ones_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_ones_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_ones_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_ones_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_new_zeros_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_alpha_dropout_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_channel_shuffle_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_channel_shuffle_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_dropout_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_elu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_gelu_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_gelu_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_group_norm_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_hardshrink_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_hardtanh_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_hardtanh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_hardtanh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_huber_loss_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_l1_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_layer_norm_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_layer_norm_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_log_softmax_with_dtype_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_margin_ranking_loss_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_margin_ranking_loss_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_margin_ranking_loss_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_margin_ranking_loss_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_mish_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_mse_loss_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_mse_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_mse_loss_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_pairwise_distance_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_pairwise_distance_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_pdist_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_pixel_shuffle_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_pixel_unshuffle_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_pixel_unshuffle_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_poisson_nll_loss_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_poisson_nll_loss_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_prelu_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_relu6_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_relu6_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_relu6_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_relu_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_smooth_l1_loss_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_softmax_with_dtype_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_softmin_with_dtype_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_softmin_with_dtype_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_softmin_with_dtype_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_softplus_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_softshrink_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_tanhshrink_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_threshold_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_threshold_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_triplet_margin_loss_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_triplet_margin_loss_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_triplet_margin_loss_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_nn_functional_triplet_margin_loss_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_norm_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_normal__in_place_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_normal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_normal_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ones_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ones_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ones_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ones_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_permute_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_permute_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_permute_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_permute_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_permute_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_permute_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_positive_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_positive_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_pow_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_pow_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_pow_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_prod_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_prod_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rad2deg_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rad2deg_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rad2deg_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ravel_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ravel_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ravel_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_ravel_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_real_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_real_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reciprocal_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reciprocal_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reciprocal_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reciprocal_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_remainder_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_remainder_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_renorm_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_renorm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_repeat_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_repeat_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_repeat_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_repeat_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_repeat_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_repeat_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reshape_as_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reshape_as_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reshape_as_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reshape_as_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reshape_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reshape_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reshape_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reshape_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_reshape_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_roll_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rot90_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rot90_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rot90_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rot90_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rot90_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rsqrt_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rsqrt_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rsqrt_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rsqrt_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rsub_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rsub_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rsub_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_rsub_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_select_scatter_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_select_scatter_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sgn_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sgn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sgn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sgn_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sgn_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sigmoid_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sigmoid_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sign_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sign_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sign_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sign_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sign_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sign_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_signbit_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_signbit_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_signbit_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_signbit_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sin_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sin_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sin_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sin_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sinc_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sinc_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sinc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sinc_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sinh_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sinh_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sinh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sinh_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sinh_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_softmax_with_dtype_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_bessel_j0_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_entr_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_entr_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_erfcx_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_erfcx_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_erfcx_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_erfcx_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_i0e_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_i0e_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_i0e_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_i0e_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_i0e_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_i1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_i1e_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_i1e_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_log_ndtr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_log_ndtr_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_log_softmax_with_dtype_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_log_softmax_with_dtype_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_log_softmax_with_dtype_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_log_softmax_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_log_softmax_with_dtype_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_logit_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_logit_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_logit_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_logit_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_multigammaln_mvlgamma_p_1_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_multigammaln_mvlgamma_p_3_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_multigammaln_mvlgamma_p_3_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_multigammaln_mvlgamma_p_3_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_multigammaln_mvlgamma_p_5_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_multigammaln_mvlgamma_p_5_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_multigammaln_mvlgamma_p_5_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_ndtr_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_ndtri_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_softmax_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_spherical_bessel_j0_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_special_spherical_bessel_j0_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sqrt_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sqrt_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sqrt_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sqrt_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_square_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_square_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_square_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_square_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_squeeze_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_squeeze_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_squeeze_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_squeeze_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_squeeze_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_squeeze_multiple_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_squeeze_multiple_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_squeeze_multiple_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_squeeze_multiple_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_stack_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_stack_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_stack_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_stack_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_std_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_std_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_std_mean_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_std_mean_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_stft_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sub_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sub_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sub_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_to_size_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_to_size_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_to_size_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_sum_to_size_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_t_copy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_t_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_t_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_take_along_dim_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_take_along_dim_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_take_along_dim_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_take_along_dim_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tan_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tan_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tan_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tanh_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tanh_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tanh_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tanh_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tensor_split_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tensor_split_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_to_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_to_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_trace_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_trace_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_trace_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_transpose_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_transpose_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_transpose_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_transpose_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_transpose_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tril_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tril_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_tril_indices_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_triu_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_triu_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_triu_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_triu_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_true_divide_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_true_divide_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_true_divide_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_true_divide_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_true_divide_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_true_divide_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_trunc_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unbind_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unbind_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unflatten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_copy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_copy_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_copy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unfold_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unsqueeze_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unsqueeze_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unsqueeze_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unsqueeze_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unsqueeze_copy_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_unsqueeze_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_vdot_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_as_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_as_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_as_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_as_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_as_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_copy_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_view_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_vsplit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_vstack_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_vstack_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_vstack_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_vstack_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_zeros_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_zeros_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_meta__refs_zeros_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_T_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_T_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_T_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_bfloat16_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_bfloat16_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_bfloat16_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_bool_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_bool_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_bool_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_bool_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_byte_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_byte_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_cdouble_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_cdouble_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_cfloat_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_cfloat_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_cfloat_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_chalf_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_chalf_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_chalf_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_chalf_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_char_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_char_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_complex_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_double_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_double_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_double_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_double_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_double_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_float_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_float_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_half_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_half_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_half_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_half_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_half_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_int_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_int_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_long_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_polar_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_short_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_short_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs__conversions_short_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_abs_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_abs_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_abs_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_acos_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_acos_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_acos_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_acos_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_acosh_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_acosh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_add_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_add_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_addcdiv_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_addcmul_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_addcmul_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_addcmul_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_addr_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_addr_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_addr_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_addr_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_alias_copy_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_all_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_all_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_allclose_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_amax_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_amin_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_amin_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_amin_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_amin_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_amin_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_amin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_amin_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_any_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_any_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_any_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_any_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_arange_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_arange_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_arange_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_partial_views_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_partial_views_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_partial_views_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_scatter_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_scatter_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_as_strided_scatter_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_asin_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_asin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_asin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_asin_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_asinh_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_asinh_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_asinh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_asinh_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atan2_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atan2_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atan_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atan_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atan_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atleast_1d_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atleast_1d_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atleast_1d_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atleast_2d_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atleast_3d_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atleast_3d_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_atleast_3d_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_and_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_left_shift_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_left_shift_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_not_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_not_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_or_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_or_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_or_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_right_shift_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bitwise_xor_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_block_diag_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_block_diag_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_block_diag_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_broadcast_tensors_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_broadcast_tensors_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_broadcast_tensors_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_broadcast_tensors_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_broadcast_to_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_broadcast_to_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bucketize_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_bucketize_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cat_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cat_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cat_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cat_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cat_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cauchy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cauchy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ceil_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ceil_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ceil_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_chunk_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_chunk_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_chunk_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_chunk_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_chunk_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_max_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_max_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_max_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_max_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_max_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_min_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_min_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_min_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clamp_min_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clone_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clone_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clone_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_clone_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_column_stack_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_column_stack_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_column_stack_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_column_stack_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_conj_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_conj_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_conj_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_conj_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_conj_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_conj_physical_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_constant_pad_nd_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_constant_pad_nd_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_contiguous_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_contiguous_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_contiguous_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_copysign_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_copysign_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cos_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cos_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cos_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cosh_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cosh_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_count_nonzero_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_count_nonzero_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_count_nonzero_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_count_nonzero_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cumprod_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cumprod_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cumprod_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_cumsum_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_deg2rad_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_deg2rad_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_deg2rad_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diag_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diag_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diag_embed_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diag_embed_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diag_embed_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diag_embed_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diag_embed_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diagonal_copy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diagonal_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diagonal_copy_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diagonal_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diagonal_scatter_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diagonal_scatter_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diagonal_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_diagonal_scatter_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_digamma_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_digamma_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_digamma_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_div_no_rounding_mode_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_div_no_rounding_mode_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_div_no_rounding_mode_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_div_trunc_rounding_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_dsplit_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_like_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_like_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_like_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_strided_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_strided_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_empty_strided_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_eq_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_eq_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_equal_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_equal_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_erf_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_erf_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_erfc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_erfc_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_erfc_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_erfc_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_erfinv_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exp2_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exp2_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exp2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exp_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exp_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exp_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exp_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exp_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_as_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_as_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_as_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_copy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_copy_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_copy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expand_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expm1_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expm1_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expm1_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expm1_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_expm1_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exponential_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_exponential_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_eye_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_eye_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_eye_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fft2_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fft2_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fft2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fft2_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fft2_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fft2_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fft_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fftn_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fftn_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fftn_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fftshift_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_fftshift_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_hfft2_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_hfft_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_hfft_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_hfft_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_hfftn_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_hfftn_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ifft2_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ifft2_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ifft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ifft_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ifftn_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ifftshift_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ifftshift_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ifftshift_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft2_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft2_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfft_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_ihfftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_irfft2_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_irfft2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_irfft_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_irfft_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfft2_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfft2_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfft2_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfft_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfft_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfft_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfftn_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fft_rfftn_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fill_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fill_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fill_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fill_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flatten_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flatten_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flatten_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flatten_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flip_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flip_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flip_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flip_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flip_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flip_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fliplr_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fliplr_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fliplr_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fliplr_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flipud_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flipud_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_flipud_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_float_power_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_float_power_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_floor_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_floor_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_floor_divide_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_floor_divide_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmax_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmax_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmax_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmin_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmin_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmin_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmin_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmod_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmod_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_fmod_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_frexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ge_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ge_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_geometric_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_gt_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_gt_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_heaviside_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_hsplit_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_hsplit_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_hsplit_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_hstack_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_hstack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_hstack_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_hstack_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_hstack_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_hstack_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_i0_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_i0_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_igammac_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_imag_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_add_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_copy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_copy_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_fill_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_fill_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_select_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_index_select_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isclose_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isclose_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isclose_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isclose_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isfinite_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isfinite_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isfinite_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isinf_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isinf_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isinf_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isinf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isinf_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isnan_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isnan_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isnan_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isnan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isneginf_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isneginf_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isposinf_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isposinf_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isposinf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isposinf_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isreal_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_isreal_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_istft_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_istft_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_item_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_item_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_item_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_item_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_item_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_lcm_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_lcm_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_le_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_le_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_le_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_le_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_lerp_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_lerp_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_lerp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_lgamma_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_cross_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_cross_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_diagonal_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_diagonal_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_diagonal_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_matrix_norm_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_matrix_norm_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_svd_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_svdvals_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_vecdot_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_vecdot_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linalg_vector_norm_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linspace_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linspace_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_linspace_tensor_overload_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log10_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log10_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log10_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log10_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log10_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log1p_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log1p_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log1p_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log1p_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log_normal_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log_softmax_with_dtype_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_log_softmax_with_dtype_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logaddexp2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logaddexp2_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logaddexp_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logaddexp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logaddexp_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_and_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_and_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_not_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_not_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_not_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_or_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_or_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_xor_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_xor_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_xor_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_xor_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_xor_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_xor_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logical_xor_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logspace_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logspace_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logspace_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logspace_tensor_overload_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logspace_tensor_overload_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logspace_tensor_overload_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logspace_tensor_overload_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logsumexp_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_logsumexp_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_lt_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_lt_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_lt_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_masked_fill_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_masked_fill_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_masked_fill_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_maximum_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_maximum_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_mean_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_meshgrid_list_of_tensors_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_meshgrid_list_of_tensors_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_meshgrid_list_of_tensors_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_meshgrid_list_of_tensors_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_meshgrid_variadic_tensors_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_meshgrid_variadic_tensors_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_minimum_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_minimum_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_minimum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_minimum_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_movedim_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_movedim_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_mul_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_mul_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_mul_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_mul_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nan_to_num_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_narrow_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_narrow_copy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_narrow_copy_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_narrow_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_narrow_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_narrow_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_narrow_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_native_layer_norm_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ne_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_neg_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_neg_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_empty_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_empty_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_empty_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_empty_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_empty_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_empty_strided_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_empty_strided_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_empty_strided_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_empty_strided_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_full_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_full_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_full_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_ones_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_ones_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_ones_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_ones_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_zeros_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_zeros_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_new_zeros_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nextafter_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_alpha_dropout_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_alpha_dropout_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_celu_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_channel_shuffle_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_channel_shuffle_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_channel_shuffle_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_channel_shuffle_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_channel_shuffle_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_channel_shuffle_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_dropout_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_dropout_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_glu_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_hardtanh_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_hardtanh_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_hinge_embedding_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_l1_loss_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_l1_loss_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_l1_loss_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_layer_norm_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_layer_norm_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_log_softmax_with_dtype_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_log_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_log_softmax_with_dtype_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_margin_ranking_loss_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_mish_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_mse_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_nll_loss_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_nll_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pairwise_distance_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pairwise_distance_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pdist_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pdist_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pixel_shuffle_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pixel_shuffle_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pixel_shuffle_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pixel_shuffle_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pixel_unshuffle_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pixel_unshuffle_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pixel_unshuffle_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_pixel_unshuffle_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_poisson_nll_loss_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_prelu_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_prelu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_prelu_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_relu6_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_relu6_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_smooth_l1_loss_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_softmax_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_softmin_with_dtype_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_softmin_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_softmin_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_softmin_with_dtype_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_tanhshrink_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_tanhshrink_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_tanhshrink_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_threshold_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_threshold_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_threshold_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_threshold_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_triplet_margin_loss_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_nn_functional_triplet_margin_loss_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_normal__in_place_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_normal_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ones_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ones_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ones_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ones_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ones_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_permute_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_permute_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_permute_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_permute_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_permute_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_permute_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_permute_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_positive_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_pow_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_pow_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_prod_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_prod_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_prod_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_prod_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rad2deg_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rad2deg_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rad2deg_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rad2deg_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_randn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_randn_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ravel_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ravel_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ravel_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_ravel_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_real_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_reciprocal_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_reciprocal_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_remainder_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_renorm_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_renorm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_repeat_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_repeat_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_repeat_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_repeat_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_reshape_as_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_reshape_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_reshape_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_reshape_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_roll_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_roll_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_roll_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rot90_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rot90_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rot90_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rot90_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rot90_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_round_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_round_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rsqrt_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rsqrt_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rsqrt_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rsub_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rsub_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rsub_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rsub_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_rsub_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_select_scatter_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_select_scatter_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_select_scatter_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sgn_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sgn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sgn_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sgn_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sgn_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sgn_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sigmoid_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sigmoid_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_signbit_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_signbit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_signbit_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_signbit_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sin_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sin_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sinc_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sinc_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sinc_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sinc_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sinh_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_softmax_with_dtype_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_softmax_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_softmax_with_dtype_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_softmax_with_dtype_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_softmax_with_dtype_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_bessel_j0_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_bessel_j1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_entr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_entr_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_erfcx_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_erfcx_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_erfcx_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_erfcx_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_i0e_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_i1_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_i1_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_i1e_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_log_ndtr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_log_ndtr_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_log_ndtr_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_log_softmax_with_dtype_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_log_softmax_with_dtype_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_log_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_log_softmax_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_log_softmax_with_dtype_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_log_softmax_with_dtype_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_logit_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_logit_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_logit_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_logit_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_multigammaln_mvlgamma_p_1_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_multigammaln_mvlgamma_p_1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_multigammaln_mvlgamma_p_3_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_multigammaln_mvlgamma_p_3_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_multigammaln_mvlgamma_p_5_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_multigammaln_mvlgamma_p_5_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_multigammaln_mvlgamma_p_5_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_multigammaln_mvlgamma_p_5_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_ndtr_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_ndtr_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_ndtri_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_ndtri_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_ndtri_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_ndtri_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_softmax_with_dtype_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_softmax_with_dtype_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_softmax_with_dtype_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_softmax_with_dtype_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_softmax_with_dtype_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_spherical_bessel_j0_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_spherical_bessel_j0_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_xlog1py_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_xlog1py_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_xlog1py_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_xlog1py_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_special_zeta_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sqrt_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sqrt_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sqrt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sqrt_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_squeeze_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_squeeze_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_squeeze_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_squeeze_multiple_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_squeeze_multiple_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_squeeze_multiple_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_squeeze_multiple_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_stack_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_stack_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_stack_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_std_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_std_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_stft_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sub_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sub_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sub_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sum_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sum_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sum_to_size_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sum_to_size_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_sum_to_size_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_t_copy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_t_copy_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_t_copy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_t_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_t_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_t_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_t_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_take_along_dim_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_take_along_dim_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_take_along_dim_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tan_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tan_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tan_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tan_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tanh_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tanh_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tanh_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tensor_split_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_to_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_to_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_to_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_to_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_trace_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_transpose_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_transpose_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tril_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tril_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_tril_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_triu_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_triu_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_triu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_triu_cuda_int32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_true_divide_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_true_divide_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_true_divide_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_true_divide_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_trunc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_trunc_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unflatten_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unfold_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unfold_copy_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unfold_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unsqueeze_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unsqueeze_copy_cuda_complex32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unsqueeze_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unsqueeze_copy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unsqueeze_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unsqueeze_copy_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unsqueeze_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_unsqueeze_cuda_float32, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_var_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_vdot_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_view_as_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_view_as_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_view_copy_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_view_copy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_view_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_view_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_view_cuda_complex128, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_view_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_vsplit_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_vsplit_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_vstack_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_vstack_cuda_int8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_where_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_where_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_where_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_where_cuda_uint8, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_xlogy_cuda_bfloat16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_xlogy_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_xlogy_cuda_int16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_xlogy_cuda_int64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_zeros_cuda_bool, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_zeros_cuda_float16, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_zeros_cuda_float64, test/test_ops.py::TestCommonCUDA::test_python_ref_torch_fallback__refs_zeros_cuda_int8, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_H_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_T_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager___getitem___cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager___rdiv___cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager___rmatmul___cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager___rpow___cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager__chunk_cat_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager__unsafe_masked_index_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager__unsafe_masked_index_put_accumulate_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager__unsafe_masked_index_put_accumulate_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager__upsample_bilinear2d_aa_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_add_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_add_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_addmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_amin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_aminmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_argwhere_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_argwhere_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_as_strided_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_as_strided_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_as_strided_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_asin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_atan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_atleast_1d_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_atleast_2d_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_atleast_2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_atleast_3d_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_block_diag_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_block_diag_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_bool_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_broadcast_tensors_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_broadcast_tensors_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_broadcast_to_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_bucketize_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cartesian_prod_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cauchy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cdist_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cfloat_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_chalf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_char_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cholesky_inverse_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cholesky_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_clamp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_clamp_max_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_column_stack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_combinations_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_conj_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_constant_pad_nd_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_constant_pad_nd_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_corrcoef_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cosh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cov_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cross_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cummax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cummin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cumprod_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_cumulative_trapezoid_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_deg2rad_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_diagflat_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_diagonal_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_div_no_rounding_mode_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_dot_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_dstack_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_empty_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_empty_strided_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_empty_strided_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_eq_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_exp2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_exp_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_exp_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_expand_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_expm1_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_expm1_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fft_fftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fft_fftshift_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fft_hfft2_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fft_hfftn_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fft_ifft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fft_ifftn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fft_irfft2_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fft_irfft_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fft_irfftn_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fill_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_flipud_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_floor_divide_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_fmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_gather_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_ge_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_gradient_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_half_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_hstack_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_index_add_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_index_select_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_inner_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_int_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_isfinite_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_isinf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_istft_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_jiterator_4inputs_with_extra_args_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_jiterator_binary_return_by_ref_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_jiterator_unary_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_kthvalue_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_cond_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_det_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_det_singular_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_diagonal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_eig_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_eig_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_eigh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_eigvalsh_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_eigvalsh_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_householder_product_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_inv_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_inv_ex_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_ldl_factor_ex_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_lstsq_grad_oriented_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_lu_factor_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_lu_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_matrix_norm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_matrix_power_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_matrix_rank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_matrix_rank_hermitian_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_norm_subgradients_at_zero_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_qr_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_solve_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_solve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_solve_triangular_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_tensorinv_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linalg_tensorsolve_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linspace_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_linspace_tensor_overload_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_log2_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_logical_not_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_logspace_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_logspace_tensor_overload_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_logspace_tensor_overload_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_long_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_long_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_lu_solve_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_lu_unpack_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_mT_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_amax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_amin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_argmin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_cumsum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_mean_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_softmax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_std_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_sum_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_sum_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_masked_var_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_matmul_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_max_reduction_with_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_meshgrid_variadic_tensors_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_min_reduction_no_dim_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_msort_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nanmedian_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nansum_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_narrow_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_ne_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_new_full_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_new_zeros_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nextafter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_adaptive_avg_pool2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_adaptive_max_pool2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_alpha_dropout_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_celu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_conv1d_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_conv2d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_fractional_max_pool3d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_glu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_huber_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_interpolate_area_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_interpolate_nearest-exact_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_interpolate_nearest_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_l1_loss_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_leaky_relu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_logsigmoid_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_max_unpool1d_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_pad_constant_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_pad_reflect_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_pad_replicate_negative_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_pixel_shuffle_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_pixel_unshuffle_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_relu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_silu_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_softmin_with_dtype_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_softsign_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_tanhshrink_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_threshold_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_unfold_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_norm_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_norm_inf_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_normal_in_place_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_ones_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_ones_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_outer_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_pca_lowrank_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_pca_lowrank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_pinverse_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_polar_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_polygamma_polygamma_n_3_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_positive_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_prod_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_qr_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_quantile_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_randn_like_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_ravel_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_real_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_reciprocal_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_renorm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_reshape_as_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_reshape_as_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_resize_as__cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_round_decimals_3_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_round_decimals_neg_3_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_rsqrt_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_rsub_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_scatter_reduce_amax_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_scatter_reduce_amin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_scatter_reduce_mean_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_searchsorted_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_select_scatter_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_signal_windows_hann_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_signal_windows_kaiser_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_signal_windows_nuttall_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_sin_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_sinc_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_sort_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_sparse_sampled_addmm_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_sparse_sampled_addmm_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_special_airy_ai_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_special_chebyshev_polynomial_u_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_special_shifted_chebyshev_polynomial_t_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_special_shifted_chebyshev_polynomial_v_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_split_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_split_list_args_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_split_with_sizes_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_square_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_sum_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_sum_to_size_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_svd_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_svd_lowrank_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_t_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_take_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_tan_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_torch_ops_aten__efficient_attention_forward_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_torch_ops_aten__safe_softmax_default_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_trace_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_transpose_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_trapezoid_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_trapezoid_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_unflatten_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_unfold_copy_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_unique_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_unsafe_chunk_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_unsafe_chunk_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_var_mean_unbiased_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_view_copy_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_vsplit_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_where_cuda_float32, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_zero__cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_zeros_cuda_complex64, test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_zeros_like_cuda_complex64, test/test_ops.py::TestCompositeComplianceCUDA::test_backward___getitem___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward___rmod___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward__batch_norm_with_update_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_addcmul_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_addmm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_amax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_amin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_asinh_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_bfloat16_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_chalf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_cholesky_inverse_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_clone_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_complex_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_constant_pad_nd_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_cov_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_cross_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_cummax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_cumsum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_cumulative_trapezoid_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_diff_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_digamma_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_div_floor_rounding_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_dsplit_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_dstack_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_einsum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_erf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_exp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_expand_copy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_expm1_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_fft_fftshift_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_fft_ifftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_fft_ihfftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_fft_rfft_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_flatten_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_flip_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_fliplr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_frac_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_frexp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_grid_sampler_2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_index_reduce_prod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_cholesky_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_cholesky_ex_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_lstsq_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_lstsq_grad_oriented_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_lu_solve_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_matrix_power_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_multi_dot_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_pinv_singular_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_svdvals_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_linalg_tensorsolve_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_logdet_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_lu_unpack_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_masked_amin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_masked_cumprod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_masked_log_softmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_masked_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_masked_softmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_matrix_exp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_max_reduction_no_dim_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_meshgrid_list_of_tensors_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_min_binary_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_min_reduction_with_dim_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_minimum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_mm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_mul_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_mvlgamma_mvlgamma_p_5_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nan_to_num_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nanmedian_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_binary_cross_entropy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_embedding_bag_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_embedding_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_gaussian_nll_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_grid_sample_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_hardswish_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_hinge_embedding_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_interpolate_linear_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_local_response_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_logsigmoid_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_max_unpool1d_grad_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_multi_margin_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_rrelu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_softmin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_softplus_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_softsign_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_nn_functional_unfold_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_norm_fro_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_normal_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_permute_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_qr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_reciprocal_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_remainder_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_renorm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_repeat_interleave_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_rot90_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_round_decimals_3_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_rsqrt_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_scatter_add_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_scatter_reduce_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_sign_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_slice_scatter_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_sparse_sampled_addmm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_special_polygamma_special_polygamma_n_0_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_split_with_sizes_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_sqrt_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_std_unbiased_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_sum_to_size_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_torch_ops_aten__efficient_attention_forward_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_triu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_unbind_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_unfold_copy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_unsafe_chunk_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_view_as_complex_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_vstack_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_backward_where_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_H_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_T_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input___getitem___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input___rmatmul___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input___rmod___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input__native_batch_norm_legit_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input__segment_reduce_offsets_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_abs_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_addmm_decomposed_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_addmv_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_aminmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_any_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_as_strided_copy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_atleast_2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_baddbmm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_bmm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_cauchy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_chalf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_clamp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_conj_physical_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_count_nonzero_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_cross_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_cummax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_cumprod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_diag_embed_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_diagonal_scatter_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_erf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_erfc_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_exp2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_expand_as_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_expm1_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_fft_fft2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_fft_hfft2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_fft_ihfftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_fft_irfft_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_fft_irfftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_fft_rfft2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_fliplr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_float_power_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_fmin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_full_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_grid_sampler_2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_gt_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_half_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_index_add_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_index_put_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_index_reduce_amax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_index_reduce_prod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_isinf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_isnan_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_kron_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_ldexp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_le_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_det_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_eig_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_inv_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_ldl_factor_ex_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_lu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_lu_factor_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_lu_solve_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_matrix_rank_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_matrix_rank_hermitian_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_pinv_hermitian_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_slogdet_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_solve_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_solve_triangular_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_svdvals_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_linalg_vector_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_log10_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_log_normal_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_log_softmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_logical_or_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_lt_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_lu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_lu_unpack_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_masked_prod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_masked_std_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_masked_sum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_meshgrid_variadic_tensors_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_min_binary_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_movedim_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nanmean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nanquantile_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_narrow_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_native_batch_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nextafter_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_adaptive_avg_pool1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_avg_pool3d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_channel_shuffle_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_conv_transpose3d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_cosine_similarity_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_elu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_embedding_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_gaussian_nll_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_glu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_grid_sample_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_hinge_embedding_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_interpolate_linear_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_interpolate_trilinear_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_kl_div_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_local_response_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_margin_ranking_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_max_unpool2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_multilabel_margin_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_pad_reflect_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_pixel_shuffle_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_pixel_unshuffle_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_relu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_rms_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_selu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_soft_margin_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_softmin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_softshrink_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_softsign_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_nonzero_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_normal_in_place_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_normal_number_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_permute_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_polygamma_polygamma_n_3_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_polygamma_polygamma_n_4_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_qr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_randn_like_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_renorm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_repeat_interleave_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_reshape_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_resize_as__cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_rot90_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_scalar_tensor_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_scatter_reduce_amax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_short_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_sigmoid_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_signal_windows_cosine_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_signal_windows_gaussian_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_signal_windows_general_hamming_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_signal_windows_kaiser_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_sparse_mm_reduce_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_special_chebyshev_polynomial_w_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_special_entr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_special_hermite_polynomial_h_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_special_i1e_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_special_legendre_polynomial_p_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_special_polygamma_special_polygamma_n_0_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_special_xlog1py_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_split_list_args_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_split_with_sizes_copy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_sqrt_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_std_mean_unbiased_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_sub_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_torch_ops_aten__safe_softmax_default_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_triangular_solve_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_tril_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_unbind_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_unique_consecutive_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_var_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_var_mean_unbiased_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_xlogy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_cow_input_zeros_like_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_H_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_T_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad___radd___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad___rmatmul___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_addcmul_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_addmm_decomposed_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_all_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_atleast_1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_baddbmm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_bfloat16_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_cdouble_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_chalf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_char_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_contiguous_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_cosh_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_cummin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_cumprod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_div_trunc_rounding_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_dot_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_einsum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_eq_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_erf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_erfinv_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_expand_as_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_exponential_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fft_fftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fft_hfft_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fft_ifft_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fft_ihfft_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fft_ihfftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fft_irfft2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fft_rfftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fill_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fliplr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_fmin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_ge_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_geometric_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_half_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_hstack_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_igamma_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_index_fill_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_inner_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_isneginf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_isreal_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_kron_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_ldexp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_le_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_cholesky_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_diagonal_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_eigvals_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_inv_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_lstsq_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_lu_factor_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_qr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_slogdet_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_tensorinv_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_linalg_vector_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_logaddexp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_logspace_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_logspace_tensor_overload_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_logsumexp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_lt_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_masked_cumsum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_masked_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_masked_select_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_masked_softmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_masked_softmin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_masked_sum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_matrix_exp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_meshgrid_list_of_tensors_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_multinomial_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nansum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_narrow_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_native_batch_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_neg_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_new_full_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_new_zeros_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nextafter_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_alpha_dropout_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_batch_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_conv1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_conv2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_conv_transpose1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_feature_alpha_dropout_without_train_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_hardshrink_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_hinge_embedding_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_interpolate_nearest_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_l1_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_linear_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_max_pool2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_max_unpool1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_max_unpool1d_grad_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_pixel_unshuffle_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_rms_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_smooth_l1_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_soft_margin_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_softplus_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_softshrink_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_tanhshrink_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_nn_functional_upsample_bilinear_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_normal_number_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_ones_like_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_outer_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_polygamma_polygamma_n_4_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_prod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_put_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_quantile_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_rand_like_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_ravel_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_real_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_round_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_scatter_reduce_amax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_scatter_reduce_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_searchsorted_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_select_scatter_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_sgn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_signal_windows_blackman_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_signal_windows_cosine_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_signal_windows_gaussian_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_signal_windows_general_cosine_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_signal_windows_hann_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_softmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_sparse_mm_reduce_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_sparse_sampled_addmm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_special_bessel_j1_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_special_chebyshev_polynomial_t_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_special_chebyshev_polynomial_w_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_special_entr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_special_legendre_polynomial_p_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_special_modified_bessel_i0_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_special_modified_bessel_k1_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_special_zeta_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_split_list_args_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_squeeze_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_squeeze_multiple_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_std_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_std_mean_unbiased_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_std_unbiased_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_sub_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_sum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_sum_to_size_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_t_copy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_tanh_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_tensordot_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_tile_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_to_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_topk_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_trunc_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_unflatten_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_unique_consecutive_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_unique_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_unsqueeze_copy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_unsqueeze_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_vdot_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_view_copy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_vsplit_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_vstack_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_xlogy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_forward_ad_zeros_like_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator___rmod___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator__chunk_cat_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator__native_batch_norm_legit_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_add_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_addbmm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_addr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_all_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_amax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_amin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_aminmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_any_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_argmin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_atanh_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_bernoulli_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_ceil_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_chalf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_clamp_min_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_clone_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_corrcoef_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_cos_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_cummin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_diag_embed_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_div_floor_rounding_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_double_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_dsplit_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_dstack_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_empty_permuted_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_eq_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_erfinv_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_expand_as_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_exponential_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_eye_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_fft_fft_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_fft_hfft2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_fft_ifftshift_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_fft_ihfft_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_fft_ihfftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_flipud_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_frexp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_ge_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_geometric_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_gradient_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_heaviside_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_hstack_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_igammac_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_index_add_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_index_fill_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_index_reduce_amax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_index_reduce_amin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_index_select_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_isreal_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_jiterator_binary_return_by_ref_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_kron_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_lerp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_lgamma_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_diagonal_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_eigvals_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_ldl_factor_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_lstsq_grad_oriented_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_lu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_lu_factor_ex_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_solve_triangular_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_svd_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_tensorsolve_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linalg_vecdot_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linspace_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_linspace_tensor_overload_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_log10_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_log_softmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_logaddexp2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_logical_not_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_logical_xor_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_mH_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_masked_amax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_masked_cumsum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_masked_fill_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_masked_logaddexp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_masked_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_masked_softmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_max_reduction_with_dim_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_meshgrid_list_of_tensors_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_meshgrid_variadic_tensors_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_minimum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_mvlgamma_mvlgamma_p_3_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nanmean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_native_dropout_backward_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_new_empty_strided_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_adaptive_avg_pool1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_adaptive_max_pool3d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_alpha_dropout_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_avg_pool1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_avg_pool2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_batch_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_batch_norm_without_cudnn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_bilinear_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_binary_cross_entropy_with_logits_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_fractional_max_pool3d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_gelu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_hardshrink_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_hardsigmoid_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_hardswish_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_interpolate_linear_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_interpolate_nearest-exact_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_linear_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_logsigmoid_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_margin_ranking_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_max_unpool1d_grad_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_multilabel_soft_margin_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_pad_constant_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_pad_replicate_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_pairwise_distance_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_pixel_shuffle_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_rms_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_smooth_l1_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_soft_margin_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nn_functional_tanhshrink_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nonzero_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_nonzero_static_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_norm_nuc_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_ones_like_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_pca_lowrank_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_permute_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_pinverse_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_polygamma_polygamma_n_2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_polygamma_polygamma_n_4_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_qr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_quantile_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_rad2deg_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_randint_like_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_real_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_reciprocal_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_reshape_as_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_reshape_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_resolve_conj_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_round_decimals_0_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_rsub_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_scatter_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_scatter_reduce_amax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_scatter_reduce_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_scatter_reduce_prod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_select_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_select_scatter_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_signal_windows_blackman_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_signal_windows_nuttall_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_sin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_slice_scatter_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_special_bessel_j1_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_special_bessel_y0_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_special_chebyshev_polynomial_u_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_special_entr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_special_log_ndtr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_special_polygamma_special_polygamma_n_0_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_special_scaled_modified_bessel_k0_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_special_shifted_chebyshev_polynomial_v_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_special_zeta_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_split_with_sizes_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_std_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_sum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_svd_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_take_along_dim_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_tanh_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_tile_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_to_sparse_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_torch_ops_aten__efficient_attention_forward_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_torch_ops_aten__safe_softmax_default_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_trapezoid_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_trapz_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_var_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_operator_var_unbiased_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_H_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay___rmul___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay___rpow___cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay__batch_norm_with_update_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay__segment_reduce_lengths_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay__unsafe_masked_index_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_addmv_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_amax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_angle_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_arange_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_as_strided_partial_views_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_asin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_asinh_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_atan_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_atleast_2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_bernoulli_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_broadcast_tensors_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_cdouble_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_chunk_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_clamp_min_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_conj_physical_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_contiguous_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_cos_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_cummin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_cumsum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_diag_embed_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_div_floor_rounding_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_double_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_empty_permuted_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_empty_strided_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_expand_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_exponential_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fft_fftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fft_ifftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fft_ifftshift_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fft_ihfft_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fft_irfft2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fft_irfftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fft_rfft_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fft_rfftn_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fill_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_floor_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_fmod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_ge_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_geqrf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_index_add_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_index_fill_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_index_reduce_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_index_select_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_int_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_isposinf_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_jiterator_2inputs_2outputs_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_jiterator_unary_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_le_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_lerp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_cholesky_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_cross_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_det_singular_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_ldl_factor_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_lstsq_grad_oriented_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_lu_factor_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_lu_solve_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_matrix_power_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_matrix_rank_hermitian_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_multi_dot_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_pinv_hermitian_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linalg_svdvals_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_linspace_tensor_overload_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_log1p_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_log2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_logical_not_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_masked_argmin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_masked_fill_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_masked_log_softmax_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_masked_logsumexp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_masked_normalize_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_masked_softmin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_matmul_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_matrix_exp_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_max_pool2d_with_indices_backward_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_max_reduction_no_dim_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_maximum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_mean_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_meshgrid_variadic_tensors_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_min_binary_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_min_reduction_with_dim_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_mv_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_mvlgamma_mvlgamma_p_1_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nanmedian_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_new_empty_strided_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_adaptive_avg_pool1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_adaptive_max_pool3d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_avg_pool1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_avg_pool2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_conv1d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_dropout3d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_embedding_bag_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_embedding_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_gelu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_group_norm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_hinge_embedding_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_interpolate_area_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_interpolate_nearest-exact_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_interpolate_nearest_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_l1_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_margin_ranking_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_max_pool2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_max_unpool2d_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_mish_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_multi_head_attention_forward_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_multi_margin_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_multilabel_margin_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_multilabel_soft_margin_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_nll_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_pad_replicate_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_pairwise_distance_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_relu_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_softmin_with_dtype_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_nn_functional_upsample_nearest_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_outer_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_pca_lowrank_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_polygamma_polygamma_n_2_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_pow_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_rand_like_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_renorm_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_reshape_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_resize_as__cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_round_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_round_decimals_neg_3_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_scalar_tensor_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_scatter_reduce_amin_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_scatter_reduce_prod_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_scatter_reduce_sum_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_signal_windows_bartlett_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_signal_windows_cosine_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_signal_windows_hamming_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_sparse_mm_reduce_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_special_hermite_polynomial_h_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_special_i1e_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_special_log_ndtr_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_special_ndtri_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_special_scaled_modified_bessel_k1_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_special_shifted_chebyshev_polynomial_u_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_special_shifted_chebyshev_polynomial_v_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_special_spherical_bessel_j0_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_special_xlog1py_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_std_mean_unbiased_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_svd_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_tanh_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_transpose_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_trapz_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_unsafe_split_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_vstack_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_xlogy_cuda_float32, test/test_ops.py::TestCompositeComplianceCUDA::test_view_replay_zeros_cuda_float32, test/test_ops.py::TestMathBitsCUDA::test_conj_view_T_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view___getitem___cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view___rdiv___cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view___rpow___cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs__conversions_byte_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs__conversions_cfloat_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs__conversions_int_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_addcmul_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_any_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_as_strided_copy_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_as_strided_partial_views_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_asin_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_cat_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_cumprod_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_div_no_rounding_mode_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_empty_like_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_equal_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_exp2_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_exp_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_expand_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_expm1_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_fft_hfftn_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_fft_ifft2_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_fft_ifftshift_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_fft_irfft_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_fft_irfftn_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_fill_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_flatten_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_fliplr_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_index_select_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_isclose_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_isfinite_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_isnan_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_linspace_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_linspace_tensor_overload_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_logical_not_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_mean_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_meshgrid_list_of_tensors_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_narrow_copy_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_ne_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_new_full_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_new_ones_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_nn_functional_pixel_unshuffle_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_norm_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_pow_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_prod_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_renorm_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_rsub_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_special_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_sqrt_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_square_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_stack_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_std_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_std_mean_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_sum_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_sum_to_size_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_tensor_split_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_triu_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_unflatten_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_view_as_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_vstack_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__refs_where_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__unsafe_masked_index_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view__unsafe_masked_index_put_accumulate_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_acosh_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_addcmul_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_addmm_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_as_strided_copy_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_as_strided_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_as_strided_partial_views_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_as_strided_scatter_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_baddbmm_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_block_diag_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_broadcast_to_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_cartesian_prod_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_cdouble_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_cholesky_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_cholesky_inverse_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_combinations_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_contiguous_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_cos_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_cumprod_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_diag_embed_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_diagonal_copy_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_diagonal_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_dist_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_dot_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_empty_permuted_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_exp_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_expand_copy_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_fft_fft_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_fft_hfft_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_fft_ifft_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_fft_ifftshift_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_fft_irfft_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_fill_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_flipud_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_float_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_geqrf_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_gradient_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_hstack_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_index_add_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_index_copy_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_isclose_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_isfinite_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_item_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_jiterator_binary_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linalg_det_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linalg_householder_product_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linalg_lu_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linalg_lu_factor_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linalg_lu_factor_ex_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linalg_matrix_rank_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linalg_pinv_singular_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linalg_slogdet_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linalg_vector_norm_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_linspace_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_log10_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_lu_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_masked_mean_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_masked_prod_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_masked_sum_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_masked_var_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_matrix_exp_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_movedim_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nansum_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_narrow_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_new_full_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_new_zeros_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nn_functional_conv1d_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nn_functional_conv3d_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nn_functional_conv_transpose2d_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nn_functional_feature_alpha_dropout_without_train_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nn_functional_l1_loss_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nn_functional_normalize_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nn_functional_pad_replicate_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nn_functional_silu_complex_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nn_functional_softsign_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_nonzero_static_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_norm_nuc_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_normal_in_place_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_ones_like_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_outer_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_pca_lowrank_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_permute_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_qr_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_real_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_resize__cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_resolve_neg_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_rot90_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_rsqrt_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_scalar_tensor_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_scatter_add_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_select_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_sgn_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_short_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_softmax_with_dtype_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_split_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_square_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_std_unbiased_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_stft_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_svd_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_t_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_tanh_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_tensordot_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_to_sparse_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_unbind_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_uniform_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_unsafe_split_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_var_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_vsplit_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_vstack_cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_conj_view_zero__cuda_complex64, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view___rdiv___cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view___rmatmul___cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs__conversions_int_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_addcmul_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_allclose_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_any_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_as_strided_copy_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_as_strided_partial_views_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_atanh_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_cumprod_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_diagonal_copy_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_div_no_rounding_mode_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_empty_like_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_fft_fftshift_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_fft_hfft_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_fft_irfft_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_fft_irfftn_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_hsplit_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_index_copy_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_istft_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_item_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_lerp_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_linalg_svd_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_linalg_vecdot_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_linspace_tensor_overload_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_logical_not_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_logspace_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_masked_fill_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_meshgrid_list_of_tensors_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_movedim_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_mul_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_neg_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_new_empty_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_nn_functional_l1_loss_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_nn_functional_log_softmax_with_dtype_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_nn_functional_tanhshrink_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_normal__in_place_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_ones_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_positive_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_repeat_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_rot90_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_rsqrt_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_special_log_softmax_with_dtype_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_special_softmax_with_dtype_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_square_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_squeeze_multiple_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_std_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_sum_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_t_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_tril_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_triu_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_vdot_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_vsplit_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__refs_where_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view__unsafe_masked_index_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_addbmm_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_addmm_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_addmv_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_allclose_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_any_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_atan_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_atleast_2d_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_atleast_3d_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_block_diag_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_bool_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_cdouble_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_cholesky_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_clone_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_combinations_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_cos_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_diagonal_copy_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_double_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_empty_like_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_empty_permuted_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_eq_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_exp_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_expand_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_eye_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_fft_fft2_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_fft_fft_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_fft_fftn_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_fft_hfft_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_fft_ifftn_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_fft_irfft2_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_fft_irfftn_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_fill_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_gather_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_geqrf_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_hstack_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_index_copy_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_isfinite_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_lerp_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_cross_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_det_singular_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_eig_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_ldl_factor_ex_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_lu_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_lu_factor_ex_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_solve_triangular_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_svdvals_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_vecdot_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linalg_vector_norm_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_linspace_tensor_overload_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_log2_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_logical_not_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_logical_xor_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_lu_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_lu_unpack_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_masked_scatter_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_matmul_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_mean_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_mul_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_mv_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_nansum_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_narrow_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_neg_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_new_full_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_new_ones_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_nn_functional_conv_transpose3d_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_nn_functional_l1_loss_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_nn_functional_pad_constant_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_nn_functional_pad_reflect_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_nn_functional_silu_complex_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_nn_functional_softmin_with_dtype_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_nn_functional_triplet_margin_with_distance_loss_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_outer_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_qr_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_roll_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_rsqrt_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_sqrt_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_square_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_std_mean_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_take_along_dim_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_take_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_tanh_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_tensordot_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_tile_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_tril_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_triu_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_uniform_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_unsafe_chunk_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_unsqueeze_copy_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_var_mean_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_view_as_real_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_conj_view_vsplit_cuda_complex128, test/test_ops.py::TestMathBitsCUDA::test_neg_view_H_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view___rdiv___cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view___rmul___cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_T_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs__conversions_char_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs__conversions_half_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs__conversions_polar_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_atan_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_cauchy_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_ceil_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_clamp_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_clamp_max_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_column_stack_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_conj_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_conj_physical_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_equal_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_erf_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_erfc_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fft_fft_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fft_hfft2_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fft_hfftn_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fft_ifft_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fft_ihfft_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fill_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fliplr_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_floor_divide_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fmax_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fmin_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_fmod_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_frac_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_ge_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_hsplit_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_linalg_diagonal_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_linspace_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_log1p_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_log_normal_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_logaddexp_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_logical_and_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_logical_not_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_logspace_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_lt_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_maximum_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nan_to_num_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_narrow_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_ne_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_new_zeros_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nextafter_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_celu_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_channel_shuffle_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_hardshrink_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_leaky_relu_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_margin_ranking_loss_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_pdist_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_prelu_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_relu6_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_relu_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_softmax_with_dtype_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_nn_functional_threshold_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_norm_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_prod_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_roll_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_rsqrt_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_sinh_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_special_erfcx_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_special_multigammaln_mvlgamma_p_1_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_special_ndtr_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_square_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_std_mean_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_sub_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_tan_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_transpose_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_view_copy_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__refs_zeros_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__segment_reduce_offsets_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view__upsample_bilinear2d_aa_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_addcdiv_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_any_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_as_strided_copy_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_asinh_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_atan2_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_bernoulli_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_block_diag_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_bmm_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_bool_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_broadcast_to_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_cat_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_cdouble_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_cfloat_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_chalf_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_cholesky_inverse_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_clamp_min_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_conj_physical_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_constant_pad_nd_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_contiguous_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_diag_embed_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_diagflat_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_diff_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_div_trunc_rounding_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_dot_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_eq_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_equal_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_erfc_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_erfinv_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_expand_as_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_fft_fftn_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_fft_irfftn_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_fill_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_fmin_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_fmod_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_full_like_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_gather_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_igammac_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_index_reduce_amin_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_index_select_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_int_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_isnan_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_isreal_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_linalg_cross_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_linalg_det_singular_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_linalg_eigh_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_linalg_lu_factor_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_linalg_matrix_norm_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_linalg_pinv_singular_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_linalg_qr_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_linalg_vector_norm_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_log10_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_logsumexp_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_lt_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_lu_unpack_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_masked_amin_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_masked_log_softmax_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_masked_mean_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_masked_median_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_masked_norm_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_masked_sum_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_masked_var_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_msort_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_mv_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_narrow_copy_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_new_empty_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_new_empty_strided_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_adaptive_avg_pool2d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_adaptive_avg_pool3d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_adaptive_max_pool2d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_celu_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_conv2d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_conv_transpose2d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_conv_transpose3d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_dropout2d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_dropout3d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_feature_alpha_dropout_with_train_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_gaussian_nll_loss_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_group_norm_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_hardtanh_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_hinge_embedding_loss_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_huber_loss_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_interpolate_bicubic_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_interpolate_linear_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_margin_ranking_loss_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_max_pool1d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_max_unpool2d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_max_unpool3d_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_max_unpool3d_grad_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_multilabel_margin_loss_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_nll_loss_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_relu_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_selu_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_softsign_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_nn_functional_unfold_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_norm_fro_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_normal_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_outer_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_pinverse_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_polygamma_polygamma_n_1_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_polygamma_polygamma_n_4_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_randn_like_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_ravel_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_reciprocal_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_remainder_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_repeat_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_repeat_interleave_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_rot90_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_scalar_tensor_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_scatter_reduce_amin_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_searchsorted_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_signal_windows_cosine_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_signal_windows_general_hamming_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_signal_windows_hamming_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_signbit_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_softmax_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_sort_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_sparse_mm_reduce_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_special_airy_ai_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_special_chebyshev_polynomial_w_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_special_entr_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_special_hermite_polynomial_h_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_special_i1_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_special_legendre_polynomial_p_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_special_log_ndtr_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_special_shifted_chebyshev_polynomial_t_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_special_spherical_bessel_j0_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_split_with_sizes_copy_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_sqrt_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_square_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_std_mean_unbiased_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_svd_lowrank_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_t_copy_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_t_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_tan_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_tanh_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_to_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_to_sparse_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_transpose_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_triangular_solve_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_triu_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_trunc_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_unfold_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_unique_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_var_mean_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_var_unbiased_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_vdot_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_view_as_complex_cuda_float64, test/test_ops.py::TestMathBitsCUDA::test_neg_view_xlogy_cuda_float64, test/test_ops.py::TestFakeTensorCUDA::test_fake___rxor___cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_fake_add_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_allclose_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_any_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_arange_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_as_strided_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_asin_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_T_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast___radd___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast___rmul___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast___rsub___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast__segment_reduce_lengths_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast__unsafe_masked_index_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_acosh_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_addbmm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_addmm_decomposed_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_aminmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_angle_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_argmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_as_strided_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_as_strided_scatter_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_atan2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_atanh_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_atleast_1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_atleast_2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_baddbmm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_bernoulli_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_bitwise_and_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_cdist_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_cdouble_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_cholesky_inverse_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_chunk_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_conj_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_corrcoef_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_cos_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_cummax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_cumsum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_cumulative_trapezoid_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_deg2rad_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_diagonal_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_digamma_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_div_floor_rounding_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_div_trunc_rounding_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_empty_permuted_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_exp2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_fft_ifft_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_fft_irfft2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_fft_rfft2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_fft_rfftn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_flip_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_flipud_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_floor_divide_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_fmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_fmod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_full_like_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_gather_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_gcd_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_geometric_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_histc_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_hsplit_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_igamma_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_imag_cuda_complex64, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_index_fill_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_index_put_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_index_reduce_mean_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_isclose_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_isnan_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_isposinf_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_istft_cuda_complex64, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_jiterator_binary_return_by_ref_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_cholesky_ex_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_cond_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_ldl_factor_ex_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_lstsq_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_lu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_matrix_rank_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_multi_dot_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_norm_subgradients_at_zero_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_pinv_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_pinv_singular_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_solve_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_svd_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_svdvals_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_tensorinv_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_linalg_vector_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_log1p_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_masked_logsumexp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_masked_prod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_max_pool2d_with_indices_backward_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_meshgrid_variadic_tensors_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_mode_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_movedim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_msort_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_multinomial_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_native_dropout_backward_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_new_empty_strided_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_new_zeros_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nextafter_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_adaptive_avg_pool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_adaptive_max_pool2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_bilinear_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_binary_cross_entropy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_conv2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_conv_transpose1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_conv_transpose2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_cosine_similarity_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_fractional_max_pool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_hardshrink_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_huber_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_interpolate_linear_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_interpolate_nearest_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_mse_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_multi_margin_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_pixel_shuffle_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_pixel_unshuffle_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_prelu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_normal_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_ones_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_outer_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_pca_lowrank_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_polar_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_polygamma_polygamma_n_1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_positive_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_prod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_rand_like_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_randint_like_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_randn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_resize__cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_roll_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_round_decimals_0_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_round_decimals_neg_3_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_rsub_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_scalar_tensor_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_scatter_reduce_prod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_sigmoid_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_signal_windows_exponential_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_signbit_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_sin_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_sinc_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_softmax_with_dtype_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_special_chebyshev_polynomial_t_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_special_hermite_polynomial_he_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_special_i1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_special_log_ndtr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_special_modified_bessel_i1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_special_modified_bessel_k1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_special_polygamma_special_polygamma_n_0_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_split_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_square_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_std_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_std_mean_unbiased_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_sub_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_svd_lowrank_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_t_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_take_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_tanh_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_tensor_split_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_topk_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_trace_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_triangular_solve_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_tril_indices_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_true_divide_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_trunc_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_unfold_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_unsafe_split_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_var_mean_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_view_as_complex_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_xlogy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_autocast_zeros_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_baddbmm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_bernoulli_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_bitwise_xor_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_fake_bool_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_byte_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_char_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_clone_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_combinations_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_conj_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_copysign_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_corrcoef_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_cos_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_cross_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp___radd___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp___rmatmul___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp___rmul___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp___rsub___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp__unsafe_masked_index_put_accumulate_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_acos_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_addcdiv_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_addcmul_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_as_strided_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_atan2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_atleast_1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_atleast_3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_bernoulli_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_bfloat16_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_bmm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_chunk_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_clamp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_clamp_min_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_corrcoef_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_cov_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_cross_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_cumulative_trapezoid_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_diagonal_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_digamma_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_dot_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_dsplit_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_erfc_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_expand_as_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_fft_fft2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_fft_ihfft2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_fft_ihfftn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_fft_irfft_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_fft_rfft_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_flip_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_float_power_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_floor_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_fmod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_frac_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_gather_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_grid_sampler_2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_i0_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_index_put_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_index_reduce_mean_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_index_reduce_prod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_kron_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_eigh_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_lu_factor_ex_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_matrix_power_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_norm_subgradients_at_zero_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_pinv_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_pinv_singular_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_qr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_svd_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_vander_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_linalg_vecdot_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_log_softmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_logaddexp2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_logit_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_masked_cumprod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_masked_select_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_masked_softmin_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_matrix_exp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_max_reduction_with_dim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_meshgrid_variadic_tensors_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_mode_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_movedim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_mv_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_mvlgamma_mvlgamma_p_3_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_mvlgamma_mvlgamma_p_5_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nanmean_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nansum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_native_layer_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_neg_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_adaptive_avg_pool1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_adaptive_avg_pool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_batch_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_ctc_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_elu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_embedding_bag_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_embedding_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_fractional_max_pool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_group_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_instance_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_interpolate_nearest_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_local_response_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_margin_ranking_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_max_unpool2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_max_unpool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_nll_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_normalize_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_pad_replicate_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_pairwise_distance_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_pixel_unshuffle_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_poisson_nll_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_relu6_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_scaled_dot_product_attention_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_smooth_l1_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_soft_margin_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_tanhshrink_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_nn_functional_upsample_nearest_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_norm_inf_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_normal_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_ormqr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_polygamma_polygamma_n_2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_pow_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_ravel_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_reshape_as_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_round_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_round_decimals_0_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_round_decimals_3_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_select_scatter_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_sgn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_sigmoid_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_slice_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_slice_scatter_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_special_i1e_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_special_log_ndtr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_split_with_sizes_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_squeeze_multiple_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_sub_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_t_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_t_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_tile_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_torch_ops_aten__efficient_attention_forward_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_trapz_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_triangular_solve_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_trunc_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_unbind_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_unsqueeze_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_unsqueeze_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_amp_view_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp___rmatmul___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp___rmod___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp__batch_norm_with_update_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp__native_batch_norm_legit_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_add_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_addbmm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_addcmul_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_amin_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_as_strided_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_as_strided_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_as_strided_partial_views_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_atanh_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_broadcast_to_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_cat_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_chalf_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_cholesky_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_cholesky_solve_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_chunk_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_clone_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_constant_pad_nd_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_corrcoef_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_cov_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_cumulative_trapezoid_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_diag_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_diff_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_digamma_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_div_no_rounding_mode_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_dstack_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_einsum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_erfinv_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_expand_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_fft_hfft_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_fft_ihfft_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_fft_ihfftn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_fft_rfftn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_flatten_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_fliplr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_fmin_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_frexp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_gather_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_i0_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_index_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_index_fill_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_index_select_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_inner_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_kron_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_kthvalue_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_linalg_cond_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_linalg_cross_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_linalg_diagonal_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_linalg_lstsq_grad_oriented_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_linalg_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_linalg_pinv_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_linalg_pinv_hermitian_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_linalg_qr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_linalg_slogdet_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_log_softmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_masked_fill_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_masked_log_softmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_masked_mean_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_masked_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_masked_sum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_masked_var_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_max_reduction_no_dim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_min_binary_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_min_reduction_with_dim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nanmedian_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_narrow_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_native_batch_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_native_layer_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_avg_pool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_batch_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_binary_cross_entropy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_binary_cross_entropy_with_logits_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_channel_shuffle_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_cosine_similarity_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_cross_entropy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_dropout2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_elu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_fractional_max_pool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_gaussian_nll_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_gelu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_hinge_embedding_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_interpolate_nearest-exact_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_kl_div_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_leaky_relu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_max_pool1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_max_pool2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_max_pool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_max_unpool1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_max_unpool1d_grad_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_max_unpool2d_grad_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_max_unpool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_multilabel_margin_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_multilabel_soft_margin_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_nll_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_pad_reflect_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_pad_replicate_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_scaled_dot_product_attention_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_silu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_softmin_with_dtype_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_softshrink_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_nn_functional_softsign_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_normal_number_mean_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_prod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_put_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_ravel_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_rot90_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_round_decimals_neg_3_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_rsqrt_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_scatter_reduce_prod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_scatter_reduce_sum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_sort_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_special_entr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_special_i1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_special_log_ndtr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_split_list_args_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_split_with_sizes_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_square_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_squeeze_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_sub_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_take_along_dim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_take_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_tensordot_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_tile_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_to_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_triu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_unsqueeze_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_var_mean_unbiased_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_view_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_vsplit_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_where_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_crossref_backward_no_amp_xlogy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_cumsum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_diag_embed_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_diagonal_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_div_floor_rounding_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_einsum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_empty_like_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_empty_strided_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_expand_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_expm1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_eye_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_fft_fftshift_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_fft_hfft2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_fft_hfftn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_fft_ifftn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_fft_rfft2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_float_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_float_power_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_floor_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_frexp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_full_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_gather_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_gcd_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_fake_grid_sampler_2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_igamma_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_imag_cuda_complex64, test/test_ops.py::TestFakeTensorCUDA::test_fake_int_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_istft_cuda_complex64, test/test_ops.py::TestFakeTensorCUDA::test_fake_item_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_jiterator_2inputs_2outputs_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_kron_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_lgamma_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_cond_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_det_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_eigh_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_eigvals_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_ldl_factor_ex_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_lstsq_grad_oriented_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_lu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_lu_factor_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_matrix_power_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_pinv_singular_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_solve_ex_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_svd_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_vander_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_linalg_vector_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_log_softmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_logcumsumexp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_logical_and_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_logical_or_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_logical_xor_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_argmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_logaddexp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_mean_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_median_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_scatter_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_select_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_softmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_softmin_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_sum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_masked_var_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_matmul_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_max_reduction_no_dim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_mean_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_meshgrid_list_of_tensors_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_min_reduction_with_dim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_mul_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_mvlgamma_mvlgamma_p_5_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nan_to_num_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_narrow_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_native_layer_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_neg_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_new_ones_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_adaptive_avg_pool2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_alpha_dropout_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_batch_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_bilinear_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_binary_cross_entropy_with_logits_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_conv1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_conv2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_dropout3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_embedding_bag_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_feature_alpha_dropout_without_train_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_gelu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_group_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_interpolate_bicubic_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_kl_div_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_layer_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_local_response_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_logsigmoid_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_margin_ranking_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_max_pool2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_max_unpool1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_pad_constant_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_pad_replicate_negative_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_relu6_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_relu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_smooth_l1_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_softsign_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_threshold_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_upsample_bilinear_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nn_functional_upsample_nearest_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_nonzero_static_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_norm_fro_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_ones_like_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_ormqr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_permute_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_polar_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_polygamma_polygamma_n_1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_put_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_randn_like_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_reciprocal_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_resize__cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_resolve_conj_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_round_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_round_decimals_3_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_rsub_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_scatter_reduce_amin_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_scatter_reduce_sum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_sgn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_short_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_signal_windows_blackman_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_signal_windows_kaiser_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_sparse_mm_reduce_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_sparse_sampled_addmm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_bessel_y1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_chebyshev_polynomial_w_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_i0e_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_i1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_i1e_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_legendre_polynomial_p_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_log_ndtr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_modified_bessel_i0_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_modified_bessel_k1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_shifted_chebyshev_polynomial_u_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_shifted_chebyshev_polynomial_v_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_special_zeta_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_std_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_sub_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_svd_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_svd_lowrank_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_t_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_topk_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_torch_ops_aten__efficient_attention_forward_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_true_divide_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_unbind_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_unfold_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_uniform_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_unravel_index_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_fake_unsqueeze_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_fake_zeros_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops___getitem___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops___rdiv___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops___rsub___cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops___rxor___cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_add_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_addcdiv_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_alias_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_amax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_arange_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_argmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_as_strided_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_as_strided_scatter_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_atan2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_atleast_2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_atleast_3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_baddbmm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_bitwise_or_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_broadcast_tensors_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_cartesian_prod_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_cauchy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_chalf_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_char_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_conj_physical_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_cos_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_deg2rad_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_diagflat_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_div_floor_rounding_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_div_trunc_rounding_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_dsplit_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_einsum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_exp2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_expand_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_expand_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_expm1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_fft_hfft_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_fft_ifft2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_fft_irfft2_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_fft_irfft_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_fft_rfftn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_fliplr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_flipud_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_gather_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_gcd_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_geometric_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_heaviside_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_histc_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_hypot_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_i0_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_index_add_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_index_fill_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_index_reduce_amin_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_isnan_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_isneginf_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_isposinf_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_isreal_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_istft_cuda_complex64, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_jiterator_binary_return_by_ref_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_jiterator_unary_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_kthvalue_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_lcm_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_ldexp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_le_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_linalg_diagonal_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_linalg_eigh_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_linalg_matrix_power_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_linalg_matrix_rank_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_linalg_vector_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_linspace_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_log1p_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_log_softmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_logaddexp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_logit_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_logsumexp_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_lt_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_lu_solve_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_lu_unpack_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_mT_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_masked_argmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_masked_fill_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_masked_log_softmax_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_masked_mean_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_masked_normalize_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_masked_select_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_masked_sum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_matmul_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_maximum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_meshgrid_variadic_tensors_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_min_reduction_no_dim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_min_reduction_with_dim_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nan_to_num_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_narrow_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_native_layer_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_adaptive_avg_pool1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_adaptive_avg_pool2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_adaptive_avg_pool3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_batch_norm_without_cudnn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_celu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_conv2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_conv_transpose1d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_dropout3d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_feature_alpha_dropout_without_train_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_fractional_max_pool2d_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_glu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_hardswish_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_instance_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_interpolate_area_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_interpolate_linear_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_interpolate_nearest_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_kl_div_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_layer_norm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_mse_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_multilabel_margin_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_normalize_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_pad_constant_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_pad_replicate_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_pad_replicate_negative_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_pairwise_distance_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_pixel_shuffle_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_pixel_unshuffle_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_poisson_nll_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_silu_complex_cuda_complex64, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_silu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_soft_margin_loss_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_softmin_with_dtype_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_softplus_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_softshrink_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_nn_functional_upsample_nearest_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_norm_fro_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_normal_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_outer_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_polygamma_polygamma_n_1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_polygamma_polygamma_n_4_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_rand_like_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_renorm_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_repeat_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_reshape_as_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_round_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_rsub_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_scalar_tensor_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_scatter_add_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_scatter_reduce_amin_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_select_scatter_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_sgn_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_sigmoid_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_sign_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_signal_windows_blackman_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_signal_windows_cosine_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_signal_windows_general_hamming_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_signal_windows_hann_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_signal_windows_kaiser_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_signbit_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_bessel_j1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_chebyshev_polynomial_u_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_entr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_i1e_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_laguerre_polynomial_l_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_modified_bessel_k0_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_modified_bessel_k1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_ndtr_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_ndtri_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_scaled_modified_bessel_k1_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_shifted_chebyshev_polynomial_t_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_special_shifted_chebyshev_polynomial_v_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_split_with_sizes_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_stack_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_std_mean_unbiased_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_stft_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_sum_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_sum_to_size_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_tensordot_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_tile_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_to_sparse_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_trapz_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_tril_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_triu_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_unflatten_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_unsqueeze_copy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_view_as_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_pointwise_ops_xlogy_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_arange_cuda_float16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_arange_cuda_int8, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_linspace_tensor_overload_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_linspace_tensor_overload_cuda_float64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_linspace_tensor_overload_cuda_int32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_logspace_cuda_bfloat16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_logspace_cuda_float64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_logspace_cuda_int16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_logspace_cuda_uint8, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_logspace_tensor_overload_cuda_bfloat16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_logspace_tensor_overload_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_logspace_tensor_overload_cuda_float64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_logspace_tensor_overload_cuda_int32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_ones_cuda_complex128, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_ones_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_ones_cuda_float64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout__refs_ones_cuda_int16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_arange_cuda_float16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_arange_cuda_int32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_arange_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_full_cuda_int16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_linspace_cuda_complex128, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_linspace_tensor_overload_cuda_bfloat16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_linspace_tensor_overload_cuda_float64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_linspace_tensor_overload_cuda_uint8, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_logspace_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_logspace_cuda_int16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_logspace_cuda_int8, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_logspace_tensor_overload_cuda_bfloat16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_logspace_tensor_overload_cuda_complex128, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_logspace_tensor_overload_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_logspace_tensor_overload_cuda_float64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_logspace_tensor_overload_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_ones_cuda_float32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_ones_cuda_int16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_ones_cuda_int32, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_ones_cuda_int64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_ones_cuda_int8, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_ones_cuda_uint8, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_zeros_cuda_bfloat16, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_zeros_cuda_complex64, test/test_ops.py::TestFakeTensorCUDA::test_strided_layout_zeros_cuda_int8, test/test_ops.py::TestTagsCUDA::test_tags_H_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags___getitem___cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags___rdiv___cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__chunk_cat_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_T_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs__conversions_chalf_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs__conversions_half_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs__conversions_long_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_abs_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_acosh_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_amin_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_as_strided_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_asin_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_atan2_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_bitwise_xor_cuda_int64, test/test_ops.py::TestTagsCUDA::test_tags__refs_broadcast_shapes_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_bucketize_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_cauchy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_clamp_min_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_conj_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_deg2rad_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_diagonal_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_div_no_rounding_mode_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_dot_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_erfc_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_expand_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_expand_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_fft_fft2_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_fft_fftshift_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_fft_hfft2_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_fft_hfft_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_fft_ifft2_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_fft_ifft_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_fft_irfftn_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_fft_rfft2_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_float_power_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_floor_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_floor_divide_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_fmin_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_gcd_cuda_int64, test/test_ops.py::TestTagsCUDA::test_tags__refs_ge_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_hstack_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_index_select_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_isfinite_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_isposinf_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_item_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_le_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_linspace_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_log1p_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_log_softmax_with_dtype_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_logical_xor_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_logspace_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_logspace_tensor_overload_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_minimum_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_narrow_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_ne_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_nn_functional_channel_shuffle_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_nn_functional_elu_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_nn_functional_gelu_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_nn_functional_hardtanh_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_nn_functional_leaky_relu_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_nn_functional_nll_loss_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_nn_functional_softmin_with_dtype_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_nn_functional_softplus_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_nn_functional_tanhshrink_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_norm_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_normal__in_place_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_normal_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_ones_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_permute_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_rad2deg_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_randn_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_renorm_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_signbit_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_sin_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_softmax_with_dtype_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_special_i0e_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_special_i1_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_special_i1e_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_special_logit_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_special_spherical_bessel_j0_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_special_zeta_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_sub_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_t_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_tan_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_tril_indices_cuda_int64, test/test_ops.py::TestTagsCUDA::test_tags__refs_true_divide_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_unfold_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_var_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_view_as_complex_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_view_as_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_view_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__refs_vstack_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags__unsafe_masked_index_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_alias_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_angle_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_atan_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_atleast_2d_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_baddbmm_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_bitwise_and_cuda_int64, test/test_ops.py::TestTagsCUDA::test_tags_bitwise_left_shift_cuda_int64, test/test_ops.py::TestTagsCUDA::test_tags_bitwise_or_cuda_int64, test/test_ops.py::TestTagsCUDA::test_tags_bmm_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_broadcast_shapes_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_broadcast_to_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_cdouble_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_clamp_min_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_complex_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_contiguous_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_deg2rad_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_diagonal_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_diagonal_scatter_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_einsum_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_expand_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_exponential_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_fft_fft_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_fft_ifft2_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_fft_ifftn_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_fft_ifftshift_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_fft_ihfft_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_flipud_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_float_power_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_floor_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_frac_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_geqrf_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_histc_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_hypot_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_igammac_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_index_reduce_amin_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_isclose_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_isinf_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_isnan_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_isneginf_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_lerp_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_det_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_eig_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_ldl_solve_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_lstsq_grad_oriented_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_lu_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_lu_factor_ex_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_lu_solve_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_pinv_hermitian_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_slogdet_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_svdvals_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_linalg_tensorsolve_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_log10_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_logical_and_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_logical_or_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_logical_xor_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_long_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_lt_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_lu_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_lu_solve_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_masked_amin_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_masked_cumprod_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_masked_cumsum_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_masked_fill_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_masked_select_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_masked_softmin_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_masked_sum_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_masked_var_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_matmul_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_max_pool2d_with_indices_backward_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_max_reduction_with_dim_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_meshgrid_variadic_tensors_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_min_reduction_no_dim_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_min_reduction_with_dim_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_mode_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_msort_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_mvlgamma_mvlgamma_p_3_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_narrow_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_neg_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_new_empty_strided_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_new_zeros_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_batch_norm_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_bilinear_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_binary_cross_entropy_with_logits_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_conv1d_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_feature_alpha_dropout_with_train_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_glu_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_interpolate_nearest-exact_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_layer_norm_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_local_response_norm_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_mish_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_mse_loss_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_multilabel_soft_margin_loss_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_pad_circular_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_pad_replicate_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_pairwise_distance_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_poisson_nll_loss_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_relu6_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_softmin_with_dtype_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_nn_functional_softsign_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_ormqr_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_permute_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_polygamma_polygamma_n_1_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_polygamma_polygamma_n_2_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_renorm_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_repeat_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_repeat_interleave_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_reshape_as_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_reshape_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_resize_as__cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_roll_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_round_decimals_0_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_sign_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_signal_windows_blackman_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_signal_windows_exponential_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_signal_windows_general_hamming_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_signbit_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_slice_scatter_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_sparse_mm_reduce_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_special_airy_ai_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_special_hermite_polynomial_he_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_special_modified_bessel_k0_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_special_modified_bessel_k1_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_special_ndtr_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_special_ndtri_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_special_polygamma_special_polygamma_n_0_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_special_shifted_chebyshev_polynomial_u_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_special_spherical_bessel_j0_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_split_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_split_with_sizes_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_split_with_sizes_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_squeeze_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_squeeze_multiple_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_sub_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_tensordot_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_tile_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_to_sparse_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_topk_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_torch_ops_aten__flash_attention_forward_cuda_float16, test/test_ops.py::TestTagsCUDA::test_tags_transpose_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_trapezoid_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_tril_indices_cuda_int64, test/test_ops.py::TestTagsCUDA::test_tags_triu_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_triu_indices_cuda_int64, test/test_ops.py::TestTagsCUDA::test_tags_true_divide_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_unflatten_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_unfold_copy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_uniform_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_var_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_var_mean_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_vdot_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_view_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_where_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_xlogy_cuda_float32, test/test_ops.py::TestTagsCUDA::test_tags_zeros_like_cuda_float32 2024-08-08T21:57:48.5466352Z 2024-08-08T21:57:51.5436075Z Running test_ops_fwd_gradients 2/2 ... [2024-08-08 21:57:51.543080] 2024-08-08T21:57:51.5440315Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_ops_fwd_gradients.py', '-m', 'not serial', '--shard-id=2', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:57:51.543553] 2024-08-08T21:58:39.1569506Z 2024-08-08T21:58:39.1571607Z test_ops_fwd_gradients 1/2 was successful, full logs can be found in artifacts with path test/test-reports/test_ops_fwd_gradients_1.2_5879d65d13635670_.log 2024-08-08T21:58:39.2750146Z Running 1599 items in this shard: test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_H_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_T_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___getitem___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___radd___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___radd___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rmatmul___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rmod___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rmul___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rpow___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__segment_reduce_lengths_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__segment_reduce_offsets_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__unsafe_masked_index_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__unsafe_masked_index_put_accumulate_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_abs_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_acos_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_acosh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_add_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addbmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addcdiv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addmm_decomposed_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addmv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_alias_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_alias_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_all_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_allclose_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_aminmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_angle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_angle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_any_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_argmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_argsort_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_argwhere_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_as_strided_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_as_strided_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_as_strided_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_as_strided_partial_views_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_as_strided_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_asin_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_asinh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_asinh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atan2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atan_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atan_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atanh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atleast_1d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atleast_2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_bfloat16_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_block_diag_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_bmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_bool_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_bool_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_broadcast_to_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_broadcast_to_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_byte_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cartesian_prod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cauchy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cdouble_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cfloat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_chalf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cholesky_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cholesky_inverse_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cholesky_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_clamp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_clamp_max_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_clamp_min_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_clone_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_clone_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_column_stack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_column_stack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_combinations_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_conj_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_constant_pad_nd_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_copysign_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_corrcoef_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cov_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cov_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cross_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cross_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cummax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cumprod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cumprod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cumsum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diag_embed_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diag_embed_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diagonal_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diagonal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diagonal_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diff_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_dist_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_div_no_rounding_mode_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_double_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_double_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_dsplit_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_dsplit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_empty_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_empty_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_empty_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_empty_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_empty_permuted_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_empty_strided_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_eq_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_equal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_erfc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_exp2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_exp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_expand_as_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_expand_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_expm1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_exponential_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_eye_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_eye_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_fft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_fft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_fft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_fftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_fftshift_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_fftshift_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_hfft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_hfftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_hfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ifftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ifftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ifftshift_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ihfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_irfft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_irfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_irfft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_rfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_rfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fill_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_flip_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fliplr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_flipud_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_frexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_full_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_full_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_full_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_gather_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_gather_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_gt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_half_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_histc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_hsplit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_hstack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_hstack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_hypot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_igamma_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_igammac_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_fill_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_fill_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_put_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_put_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_reduce_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_reduce_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_reduce_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_reduce_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_select_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_inner_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_inner_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_int_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_int_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isclose_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isclose_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isfinite_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isinf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isinf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isnan_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isneginf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isposinf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_istft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_2inputs_2outputs_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_4inputs_with_extra_args_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_binary_return_by_ref_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_binary_return_by_ref_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_unary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_kthvalue_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ldexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lerp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lgamma_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_cholesky_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_cholesky_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_cholesky_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_cond_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_det_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_det_singular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_eig_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_eigh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_eigh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_eigvals_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_householder_product_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_inv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_inv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_inv_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_ldl_factor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_ldl_factor_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_ldl_factor_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_ldl_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lstsq_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lstsq_grad_oriented_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lstsq_grad_oriented_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lu_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lu_factor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lu_factor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lu_factor_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lu_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lu_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_matrix_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_matrix_power_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_matrix_rank_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_matrix_rank_hermitian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_multi_dot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_multi_dot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_norm_subgradients_at_zero_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_pinv_hermitian_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_pinv_hermitian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_pinv_singular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_qr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_qr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_slogdet_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_solve_triangular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_svd_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_svd_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_svdvals_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_svdvals_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_tensorinv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_vecdot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_vecdot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_vector_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log10_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log10_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log1p_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logaddexp2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logcumsumexp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logcumsumexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logdet_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logical_and_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logical_not_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logical_xor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logspace_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logspace_tensor_overload_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logspace_tensor_overload_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logsumexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_long_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lu_unpack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mH_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mT_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_argmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_cumprod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_cumsum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_fill_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_log_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_logaddexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_logsumexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_median_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_select_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_select_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_std_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_std_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_sum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_var_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_var_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_matmul_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_matmul_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_matrix_exp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_max_pool2d_with_indices_backward_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_max_reduction_no_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_max_reduction_with_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_median_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_meshgrid_variadic_tensors_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_meshgrid_variadic_tensors_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_min_binary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_min_reduction_no_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_minimum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_movedim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_msort_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mul_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_multinomial_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mvlgamma_mvlgamma_p_1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mvlgamma_mvlgamma_p_5_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nanquantile_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nansum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nansum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_narrow_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_narrow_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_native_dropout_backward_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ne_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_neg_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_empty_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_empty_strided_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nextafter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_adaptive_avg_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_adaptive_avg_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_adaptive_max_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_avg_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_avg_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_avg_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_binary_cross_entropy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_celu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_channel_shuffle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_channel_shuffle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv1d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv3d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv_transpose1d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv_transpose1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv_transpose2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv_transpose3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_cross_entropy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_ctc_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_dropout2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_dropout_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_elu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_embedding_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_feature_alpha_dropout_with_train_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_feature_alpha_dropout_without_train_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_feature_alpha_dropout_without_train_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_fractional_max_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_gaussian_nll_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_gelu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_hardtanh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_hinge_embedding_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_interpolate_bicubic_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_interpolate_bilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_interpolate_linear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_interpolate_nearest-exact_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_l1_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_leaky_relu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_linear_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_local_response_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_margin_ranking_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_max_unpool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_max_unpool2d_grad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_max_unpool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_mse_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_multi_head_attention_forward_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_multi_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_multilabel_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_nll_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_circular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_constant_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_reflect_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_replicate_negative_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pairwise_distance_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pairwise_distance_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pdist_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pixel_shuffle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pixel_unshuffle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_poisson_nll_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_prelu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_relu6_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_rms_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_rrelu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_selu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_smooth_l1_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_softmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_softshrink_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_softsign_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_tanhshrink_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_triplet_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_triplet_margin_with_distance_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_unfold_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_unfold_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_upsample_nearest_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nonzero_static_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nonzero_static_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_norm_fro_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_norm_inf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_norm_nuc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_normal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_normal_in_place_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_normal_in_place_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ones_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ones_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ones_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ormqr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_outer_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_outer_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_pca_lowrank_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_pca_lowrank_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_permute_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_positive_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_prod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_put_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_qr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_rad2deg_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_rand_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_rand_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_randint_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_randint_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_randn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_randn_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ravel_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_real_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_reciprocal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_renorm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_renorm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_repeat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_repeat_interleave_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_reshape_as_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_reshape_as_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_reshape_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_reshape_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_resize__cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_resolve_neg_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_resolve_neg_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_roll_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_rot90_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_round_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_round_decimals_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_rsub_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scalar_tensor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scatter_add_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scatter_reduce_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scatter_reduce_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scatter_reduce_sum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_select_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_select_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sgn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sigmoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_bartlett_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_blackman_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_exponential_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_general_cosine_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_hann_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_nuttall_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signbit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sinc_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sinh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sinh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_slice_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_softmax_with_dtype_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_softmax_with_dtype_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sort_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sparse_sampled_addmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_airy_ai_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_bessel_j1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_bessel_y0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_bessel_y1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_chebyshev_polynomial_t_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_chebyshev_polynomial_v_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_entr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_i0e_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_i1e_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_legendre_polynomial_p_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_modified_bessel_k0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_modified_bessel_k1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_ndtr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_ndtri_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_scaled_modified_bessel_k0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_scaled_modified_bessel_k1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_shifted_chebyshev_polynomial_t_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_shifted_chebyshev_polynomial_v_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_shifted_chebyshev_polynomial_w_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_xlog1py_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_zeta_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_split_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_split_with_sizes_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_split_with_sizes_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_split_with_sizes_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sqrt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_square_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_squeeze_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_squeeze_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_stack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_std_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_std_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_std_mean_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_std_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_std_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_stft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_stft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sub_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_svd_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_svd_lowrank_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_t_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_t_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_take_along_dim_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_take_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tan_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tan_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tanh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tile_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_to_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_to_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_to_sparse_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_torch_ops_aten__safe_softmax_default_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_trace_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_trapezoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_trapz_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_trapz_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tril_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tril_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_triu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_true_divide_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_true_divide_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unbind_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unflatten_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unfold_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unfold_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unsafe_split_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unsafe_split_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unsqueeze_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unsqueeze_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unsqueeze_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_var_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_var_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_var_mean_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_var_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_vdot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_view_as_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_view_as_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_view_as_real_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_view_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_vsplit_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_vsplit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_vstack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_where_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_zero__cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_zero__cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_zeros_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_zeros_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_zeros_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_H_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_T_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___radd___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___radd___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rmatmul___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rmatmul___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rmul___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rpow___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rpow___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rsub___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__segment_reduce_lengths_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__segment_reduce_offsets_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__unsafe_masked_index_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_abs_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_acos_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_add_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addbmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addcdiv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addcmul_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addcmul_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addmm_decomposed_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addmm_decomposed_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_alias_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_alias_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_allclose_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_angle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_angle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_any_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_arange_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_argmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_argmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_argsort_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_argwhere_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_argwhere_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_as_strided_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_as_strided_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_as_strided_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_as_strided_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_as_strided_partial_views_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_as_strided_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_asin_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_asinh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atleast_2d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atleast_2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_baddbmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_bfloat16_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_block_diag_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_bmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_bool_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_bucketize_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_byte_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cdist_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cdouble_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cdouble_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cfloat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cfloat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_chalf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cholesky_inverse_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cholesky_inverse_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cholesky_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_chunk_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_clamp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_clone_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_column_stack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_combinations_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_complex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_conj_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_conj_physical_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_conj_physical_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_copysign_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_corrcoef_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cos_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cosh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cov_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cov_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cross_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cummax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cummin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cumulative_trapezoid_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cumulative_trapezoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diag_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diag_embed_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diagflat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diagflat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diagonal_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diagonal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diagonal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diff_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_digamma_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_dist_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_dot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_double_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_double_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_dsplit_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_dstack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_empty_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_empty_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_empty_strided_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_eq_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_equal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_erfc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_exp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_exp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_expand_as_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_expand_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_expand_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_exponential_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_fft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_fft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_fft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_fftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_fftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_fftshift_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_hfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_hfftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_hfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ifft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ifft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ifft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ifftshift_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ihfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_irfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_irfft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_irfftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_rfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fill_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fliplr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_flipud_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_float_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_float_power_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_floor_divide_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_frac_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_full_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_full_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_gradient_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_grid_sampler_2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_histc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_hsplit_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_hstack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_hstack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_hypot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_igammac_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_add_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_add_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_reduce_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_reduce_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_reduce_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_select_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_inner_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_int_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isclose_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isclose_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isfinite_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isinf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isnan_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isnan_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isneginf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_4inputs_with_extra_args_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_binary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_binary_return_by_ref_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_binary_return_by_ref_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_unary_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_kron_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_kthvalue_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ldexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lerp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lgamma_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_cholesky_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_cholesky_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_cross_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_cross_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_det_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_det_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_det_singular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_eig_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_eigh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_eigvals_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_eigvalsh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_eigvalsh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_householder_product_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_householder_product_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_inv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_inv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_inv_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_ldl_factor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_ldl_factor_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_ldl_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_ldl_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lstsq_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lstsq_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lstsq_grad_oriented_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lu_factor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lu_factor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lu_factor_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lu_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lu_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_matrix_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_matrix_power_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_matrix_power_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_matrix_rank_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_matrix_rank_hermitian_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_matrix_rank_hermitian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_multi_dot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_norm_subgradients_at_zero_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_norm_subgradients_at_zero_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_pinv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_pinv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_pinv_hermitian_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_pinv_hermitian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_pinv_singular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_pinv_singular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_qr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_slogdet_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_svd_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_svd_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_tensorinv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_tensorsolve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_vander_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_vecdot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_vecdot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_vector_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_vector_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linspace_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linspace_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linspace_tensor_overload_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linspace_tensor_overload_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log10_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log10_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log1p_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log1p_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logcumsumexp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logical_and_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logical_not_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logical_or_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logical_or_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logical_xor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logspace_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logspace_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lu_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lu_unpack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mH_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mT_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mT_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_argmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_cumprod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_cumsum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_fill_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_median_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_select_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_select_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_softmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_sum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_var_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_matrix_exp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_matrix_exp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_max_binary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_max_reduction_with_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_maximum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_meshgrid_list_of_tensors_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_min_binary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_min_reduction_with_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_minimum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mode_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_movedim_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_msort_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mul_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_multinomial_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mvlgamma_mvlgamma_p_1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mvlgamma_mvlgamma_p_5_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nanmean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nanmedian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_narrow_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_narrow_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_native_dropout_backward_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_native_layer_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ne_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_neg_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_empty_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_empty_strided_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_ones_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_zeros_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_zeros_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_adaptive_avg_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_adaptive_max_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_adaptive_max_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_adaptive_max_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_alpha_dropout_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_avg_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_avg_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_batch_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_celu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_channel_shuffle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv1d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv3d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv_transpose3d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv_transpose3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_cosine_embedding_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_cosine_similarity_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_dropout2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_dropout3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_dropout_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_feature_alpha_dropout_without_train_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_fractional_max_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_gaussian_nll_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_hardsigmoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_hardtanh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_instance_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_interpolate_nearest-exact_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_interpolate_nearest_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_l1_loss_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_linear_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_linear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_local_response_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_max_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_max_unpool1d_grad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_max_unpool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_mish_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_multi_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_multilabel_soft_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_nll_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_normalize_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_circular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_constant_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_reflect_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_reflect_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pairwise_distance_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pixel_shuffle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pixel_shuffle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pixel_unshuffle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_poisson_nll_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_prelu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_relu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_rms_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_rrelu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_selu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_silu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_smooth_l1_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_soft_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_softmin_with_dtype_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_softmin_with_dtype_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_softplus_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_tanhshrink_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_tanhshrink_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_triplet_margin_loss_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_triplet_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_unfold_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_unfold_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_upsample_bilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nonzero_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nonzero_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nonzero_static_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_norm_fro_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_norm_inf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_norm_nuc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_normal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_normal_in_place_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ones_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ones_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ones_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ormqr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ormqr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_pca_lowrank_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_polar_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_polygamma_polygamma_n_1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_positive_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_pow_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_put_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_qr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_quantile_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_rad2deg_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_rand_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_randn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_randn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_randn_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_real_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_real_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_reciprocal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_remainder_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_renorm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_repeat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_repeat_interleave_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_repeat_interleave_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_reshape_as_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_reshape_as_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_resize_as__cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_resolve_conj_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_resolve_neg_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_resolve_neg_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_rot90_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_round_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_round_decimals_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_rsqrt_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scatter_reduce_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scatter_reduce_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scatter_reduce_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scatter_reduce_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scatter_reduce_sum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_select_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sgn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_short_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_short_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sigmoid_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sign_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_bartlett_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_exponential_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_gaussian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_general_cosine_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_hann_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_kaiser_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_nuttall_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sin_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sinc_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sinc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sinh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sinh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_slice_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_slice_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_softmax_with_dtype_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_airy_ai_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_bessel_j1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_bessel_y0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_bessel_y1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_chebyshev_polynomial_t_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_chebyshev_polynomial_u_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_chebyshev_polynomial_w_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_entr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_hermite_polynomial_h_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_i0e_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_laguerre_polynomial_l_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_log_ndtr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_modified_bessel_i1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_modified_bessel_k0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_ndtri_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_scaled_modified_bessel_k1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_shifted_chebyshev_polynomial_u_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_shifted_chebyshev_polynomial_v_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_spherical_bessel_j0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_split_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_split_list_args_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_split_with_sizes_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sqrt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_square_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_squeeze_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_squeeze_multiple_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_std_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_std_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_std_mean_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_std_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_stft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sub_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_t_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_t_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_t_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_take_along_dim_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_take_along_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_take_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_take_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tan_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tanh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tensor_split_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tile_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_to_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_to_sparse_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_to_sparse_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_topk_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_trace_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_trace_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_transpose_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_trapezoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_trapz_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_triangular_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tril_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_triu_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_triu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_trunc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unbind_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unflatten_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unfold_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unfold_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unfold_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_uniform_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unique_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unsafe_split_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unsqueeze_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_var_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_var_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_var_mean_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_var_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_var_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_view_as_complex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_view_as_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_view_as_real_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_view_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_view_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_vsplit_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_vsplit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_vstack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_vstack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_where_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_where_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_xlogy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_zero__cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_zeros_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_zeros_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_zeros_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_H_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_H_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_T_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_T_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___getitem___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___radd___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rdiv___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rmatmul___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rmatmul___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rpow___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rsub___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rsub___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__segment_reduce_lengths_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__unsafe_masked_index_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__unsafe_masked_index_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_abs_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_abs_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_acos_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_acosh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_acosh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addbmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addbmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addcmul_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addmm_decomposed_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addmm_decomposed_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_allclose_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_aminmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_angle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_any_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_argmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_argmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_argsort_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_as_strided_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_as_strided_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_as_strided_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_as_strided_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_as_strided_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_asinh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atanh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atleast_2d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atleast_2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atleast_3d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_baddbmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_bernoulli_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_block_diag_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_bmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_bmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_bool_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_broadcast_to_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_bucketize_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_byte_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_byte_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cauchy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cdouble_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cfloat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_chalf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_char_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_char_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cholesky_inverse_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cholesky_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_chunk_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_clone_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_combinations_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_complex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_conj_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_conj_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_conj_physical_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_conj_physical_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_contiguous_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_copysign_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cosh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_count_nonzero_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cov_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cross_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cumprod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cumsum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cumulative_trapezoid_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cumulative_trapezoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_deg2rad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diag_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diag_embed_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diagflat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diagonal_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diagonal_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diagonal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diagonal_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_digamma_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_dist_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_div_no_rounding_mode_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_div_no_rounding_mode_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_div_trunc_rounding_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_dot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_double_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_double_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_dsplit_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_dsplit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_einsum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_einsum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_empty_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_empty_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_empty_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_empty_permuted_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_empty_permuted_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_eq_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_erfinv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_exp2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_exp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_expand_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_expand_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_expm1_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_expm1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_eye_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_eye_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_fft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_fft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_fftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_fftshift_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_hfftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ifft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ifft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ifft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ifftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ifftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ihfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_irfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_irfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_rfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_rfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fill_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_flatten_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_flip_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fliplr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_flipud_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_float_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_frac_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_full_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_full_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ge_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_geqrf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_gradient_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_gt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_half_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_half_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_heaviside_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_histc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_hsplit_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_hstack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_hstack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_hypot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_i0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_igammac_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_fill_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_fill_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_put_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_put_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_reduce_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_reduce_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_select_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isclose_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isinf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isnan_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isposinf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_item_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_4inputs_with_extra_args_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_binary_return_by_ref_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ldexp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_le_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lerp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lerp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lgamma_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_cholesky_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_cholesky_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_cholesky_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_cond_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_cond_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_cross_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_det_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_diagonal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_diagonal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_eigvals_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_householder_product_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_householder_product_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_inv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_inv_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_ldl_factor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_ldl_factor_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_ldl_factor_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_ldl_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lstsq_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lstsq_grad_oriented_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lstsq_grad_oriented_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lu_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lu_factor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lu_factor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lu_factor_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lu_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_matrix_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_matrix_power_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_matrix_power_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_matrix_rank_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_matrix_rank_hermitian_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_matrix_rank_hermitian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_multi_dot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_norm_subgradients_at_zero_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_norm_subgradients_at_zero_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_pinv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_pinv_singular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_slogdet_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_slogdet_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_solve_triangular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_solve_triangular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_svd_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_svd_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_tensorinv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_tensorinv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_tensorsolve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_vander_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_vecdot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_vector_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linspace_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log10_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log_normal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log_softmax_with_dtype_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log_softmax_with_dtype_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logcumsumexp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logcumsumexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logical_and_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logspace_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logspace_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logspace_tensor_overload_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logspace_tensor_overload_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logsumexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lu_unpack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mH_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mT_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_argmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_cumsum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_cumsum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_logsumexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_median_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_normalize_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_prod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_softmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_std_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_sum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_var_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_matmul_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_matrix_exp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_matrix_exp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_maximum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_meshgrid_variadic_tensors_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_min_binary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_min_reduction_no_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_min_reduction_with_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_minimum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_movedim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_msort_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mul_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mvlgamma_mvlgamma_p_1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mvlgamma_mvlgamma_p_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mvlgamma_mvlgamma_p_5_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nan_to_num_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nanmean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nanmean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nanmedian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nanquantile_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_native_batch_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_native_layer_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ne_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_empty_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_empty_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_full_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_ones_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_ones_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_zeros_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_adaptive_avg_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_adaptive_avg_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_adaptive_max_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_alpha_dropout_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_avg_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_avg_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_batch_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_batch_norm_without_cudnn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_channel_shuffle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_channel_shuffle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv2d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv3d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv_transpose1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv_transpose2d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv_transpose2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv_transpose3d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_ctc_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_dropout2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_dropout_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_elu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_embedding_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_feature_alpha_dropout_without_train_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_fractional_max_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_gelu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_glu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_grid_sample_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_group_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_huber_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_instance_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_interpolate_bilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_interpolate_linear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_interpolate_nearest_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_interpolate_trilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_kl_div_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_l1_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_leaky_relu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_linear_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_logsigmoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_max_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_max_unpool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_max_unpool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_max_unpool2d_grad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_max_unpool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_multilabel_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_multilabel_soft_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_nll_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_normalize_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_constant_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_replicate_negative_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_replicate_negative_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pairwise_distance_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pdist_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pixel_shuffle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pixel_unshuffle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_prelu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_relu6_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_relu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_rms_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_rrelu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_scaled_dot_product_attention_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_silu_complex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_smooth_l1_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_softmin_with_dtype_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_softshrink_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_softsign_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_tanhshrink_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_threshold_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_unfold_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nonzero_static_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_norm_fro_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_norm_inf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_normal_number_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ones_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ones_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_outer_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_pca_lowrank_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_permute_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_pinverse_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_polar_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_polygamma_polygamma_n_0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_polygamma_polygamma_n_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_positive_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_positive_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_pow_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_put_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_qr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_quantile_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_rand_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_randint_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_randint_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_randn_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ravel_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ravel_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_remainder_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_renorm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_repeat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_repeat_interleave_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_repeat_interleave_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_reshape_as_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_reshape_as_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_reshape_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_reshape_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_resize__cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_resize__cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_resize_as__cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_resolve_conj_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_resolve_conj_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_resolve_neg_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_resolve_neg_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_roll_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_roll_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_rot90_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_rot90_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_round_decimals_0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_round_decimals_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_rsqrt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_rsub_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scatter_add_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scatter_reduce_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scatter_reduce_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scatter_reduce_sum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sgn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_short_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_short_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sigmoid_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sign_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_bartlett_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_blackman_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_gaussian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_general_cosine_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_hamming_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_nuttall_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sin_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sinc_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_slice_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_slice_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_slice_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_softmax_with_dtype_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_bessel_j0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_bessel_y1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_chebyshev_polynomial_u_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_chebyshev_polynomial_v_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_entr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_erfcx_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_hermite_polynomial_h_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_i1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_legendre_polynomial_p_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_modified_bessel_i1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_modified_bessel_k0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_ndtri_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_polygamma_special_polygamma_n_0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_scaled_modified_bessel_k1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_split_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_split_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_split_list_args_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_split_list_args_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_split_with_sizes_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_split_with_sizes_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sqrt_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_squeeze_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_squeeze_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_squeeze_multiple_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_stack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_std_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_std_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_std_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_stft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_stft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sub_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_svd_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_svd_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_svd_lowrank_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_svd_lowrank_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_t_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_t_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_take_along_dim_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tan_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tan_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tensordot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tensordot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tile_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_to_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_to_sparse_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_topk_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_torch_ops_aten__safe_softmax_default_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_trace_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_transpose_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_transpose_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_trapz_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_triangular_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tril_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_true_divide_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_trunc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unflatten_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unfold_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unfold_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unfold_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unfold_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unique_consecutive_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unsafe_chunk_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unsafe_chunk_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unsafe_split_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unsafe_split_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unsqueeze_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_var_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_var_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_var_mean_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_var_mean_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_vdot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_view_as_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_view_as_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_vsplit_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_vstack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_vstack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_where_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_xlogy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_zero__cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_zero__cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_zeros_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_zeros_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_zeros_like_cuda_complex128 2024-08-08T21:58:39.3745878Z 2024-08-08T21:58:42.9824778Z Running inductor/test_cuda_cpp_wrapper 2/3 ... [2024-08-08 21:58:42.981877] 2024-08-08T21:58:42.9829875Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_cuda_cpp_wrapper.py', '-m', 'not serial', '--shard-id=2', '--num-shards=3', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:58:42.982438] 2024-08-08T21:58:50.8119841Z 2024-08-08T21:58:50.8121664Z inductor/test_cuda_cpp_wrapper 2/3 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_cuda_cpp_wrapper_2.3_89baaa21f06d7d6d_.log 2024-08-08T21:58:50.8122928Z 2024-08-08T21:58:54.6890834Z Running inductor/test_indexing 1/1 ... [2024-08-08 21:58:54.688509] 2024-08-08T21:58:54.6894950Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_indexing.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:58:54.689038] 2024-08-08T21:59:02.6203790Z 2024-08-08T21:59:02.6206546Z inductor/test_indexing 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_indexing_1.1_67c7a861bad10a91_.log 2024-08-08T21:59:02.6217171Z Running 16 items in this shard: test/inductor/test_indexing.py::TestIndexingSimplification::test_expand_floor_div_applied, test/inductor/test_indexing.py::TestIndexingSimplification::test_expand_floor_div_skipped, test/inductor/test_indexing.py::TestIndexingSimplification::test_indexing_join, test/inductor/test_indexing.py::TestIndexingSimplification::test_indexing_simplification, test/inductor/test_indexing.py::TestIndexingSimplification::test_int8_unpack, test/inductor/test_indexing.py::TestIndexingSimplification::test_modular_indexing_pairs_merged, test/inductor/test_indexing.py::TestIndexingSimplification::test_modular_indexing_pairs_not_merged, test/inductor/test_indexing.py::ExprPrinterTests::test_print_Min_Max, test/inductor/test_indexing.py::ExprPrinterTests::test_print_ceil, test/inductor/test_indexing.py::ExprPrinterTests::test_print_floor, test/inductor/test_indexing.py::ExprPrinterTests::test_print_floor_div, test/inductor/test_indexing.py::ExprPrinterTests::test_print_pow, test/inductor/test_indexing.py::ExprPrinterTests::test_print_round, test/inductor/test_indexing.py::ExprPrinterTests::test_print_round_decimal_ndigits_-1, test/inductor/test_indexing.py::ExprPrinterTests::test_print_round_decimal_ndigits_0, test/inductor/test_indexing.py::ExprPrinterTests::test_print_round_decimal_ndigits_1 2024-08-08T21:59:02.6225815Z 2024-08-08T21:59:06.5133865Z Running dynamo/test_nops 1/1 ... [2024-08-08 21:59:06.512828] 2024-08-08T21:59:06.5138840Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'dynamo/test_nops.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:59:06.513344] 2024-08-08T21:59:10.8863417Z 2024-08-08T21:59:10.8866511Z dynamo/test_nops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/dynamo.test_nops_1.1_bae74dff6d5edb0e_.log 2024-08-08T21:59:10.8870947Z Running 4 items in this shard: test/dynamo/test_nops.py::NopTests::test1, test/dynamo/test_nops.py::NopTests::test2, test/dynamo/test_nops.py::NopTests::test3, test/dynamo/test_nops.py::NopTests::test_extended_args 2024-08-08T21:59:10.8873660Z 2024-08-08T21:59:14.7132747Z Running inductor/test_triton_heuristics 1/1 ... [2024-08-08 21:59:14.712684] 2024-08-08T21:59:14.7137554Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_triton_heuristics.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:59:14.713191] 2024-08-08T21:59:18.4349539Z 2024-08-08T21:59:18.4351813Z inductor/test_triton_heuristics 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_triton_heuristics_1.1_b514c5a23e73b968_.log 2024-08-08T21:59:18.4353118Z 2024-08-08T21:59:22.3830700Z Running inductor/test_foreach 1/1 ... [2024-08-08 21:59:22.382426] 2024-08-08T21:59:22.3834656Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_foreach.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:59:22.382950] 2024-08-08T21:59:55.4638646Z 2024-08-08T21:59:55.4641873Z inductor/test_foreach 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_foreach_1.1_7fa09e884456c483_.log 2024-08-08T21:59:55.4776644Z Running 211 items in this shard: test/inductor/test_foreach.py::ForeachTests::test_2d_block_mixed_sizes_with_mask, test/inductor/test_foreach.py::ForeachTests::test_2d_block_no_mixed_sizes_no_mask, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_elems__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_elems__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_elems__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_elems__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_elems__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_elems__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_elems__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_elems__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_elems__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_mixed_sizes__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_mixed_sizes__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_mixed_sizes__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_mixed_sizes__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_mixed_sizes__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_mixed_sizes__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_mixed_sizes__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_mixed_sizes__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_2d_blocking_partitioning_mixed_sizes__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_aliasing, test/inductor/test_foreach.py::ForeachTests::test_broadcasting__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_broadcasting__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_broadcasting__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_broadcasting__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_cpu_cpp_fallback__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_cpu_cpp_fallback__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_cpu_cpp_fallback__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_cpu_cpp_fallback__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_cpu_cpp_fallback__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_cpu_cpp_fallback__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_cpu_cpp_fallback__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_cpu_cpp_fallback__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_cpu_cpp_fallback__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_decomp__foreach_addcdiv, test/inductor/test_foreach.py::ForeachTests::test_decomp__foreach_addcmul, test/inductor/test_foreach.py::ForeachTests::test_dynamic_shapes_fallback__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_dynamic_shapes_fallback__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_dynamic_shapes_fallback__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_dynamic_shapes_fallback__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_dynamic_shapes_fallback__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_dynamic_shapes_fallback__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_dynamic_shapes_fallback__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_dynamic_shapes_fallback__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_dynamic_shapes_fallback__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_foreach_cpp_wrapper_cuda, test/inductor/test_foreach.py::ForeachTests::test_fuse_concat, test/inductor/test_foreach.py::ForeachTests::test_fusion_duplicate_buffer_list__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_fusion_duplicate_buffer_list__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_fusion_duplicate_buffer_list__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_fusion_duplicate_buffer_list__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_fusion_duplicate_buffer_list__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_fusion_duplicate_buffer_list__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_fusion_duplicate_buffer_list__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_fusion_duplicate_buffer_list__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_fusion_duplicate_buffer_list__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_kernel_split_arg_limit_list__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_kernel_split_arg_limit_list__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_kernel_split_arg_limit_list__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_kernel_split_arg_limit_list__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_kernel_split_arg_limit_scalar__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_kernel_split_arg_limit_scalar__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_kernel_split_arg_limit_scalar__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_kernel_split_arg_limit_scalar__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_multi_device, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_abs, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_neg, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_reciprocal, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_sign, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_sqrt, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_list__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_abs, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_neg, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_reciprocal, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_sign, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_sqrt, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_list__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_scalar__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_scalar__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_scalar__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_producer_scalar__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_scalar__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_scalar__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_scalar__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_consumer_scalar__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_abs, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_neg, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_reciprocal, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_sign, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_sqrt, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_list__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_scalar__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_scalar__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_scalar__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_non_foreach_producer_scalar__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_reinplacing__foreach_add_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing__foreach_div_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing__foreach_mul_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing__foreach_sub_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing_mut_after__foreach_add_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing_mut_after__foreach_div_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing_mut_after__foreach_mul_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing_mut_after__foreach_sub_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing_mut_before__foreach_add_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing_mut_before__foreach_div_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing_mut_before__foreach_mul_, test/inductor/test_foreach.py::ForeachTests::test_reinplacing_mut_before__foreach_sub_, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_abs, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_neg, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_reciprocal, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_sign, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_sqrt, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_list__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_scalar__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_scalar__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_scalar__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_scheduler_fusion_scalar__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_abs, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_neg, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_reciprocal, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_sign, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_sqrt, test/inductor/test_foreach.py::ForeachTests::test_single_list__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_single_scalar__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_single_scalar__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_single_scalar__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_single_scalar__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_single_scalar_tensor__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_single_scalar_tensor__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_abs, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_neg, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_reciprocal, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_sign, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_sqrt, test/inductor/test_foreach.py::ForeachTests::test_singleton_lists__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_type_promotion__foreach_add, test/inductor/test_foreach.py::ForeachTests::test_type_promotion__foreach_clamp_max, test/inductor/test_foreach.py::ForeachTests::test_type_promotion__foreach_clamp_min, test/inductor/test_foreach.py::ForeachTests::test_type_promotion__foreach_copy, test/inductor/test_foreach.py::ForeachTests::test_type_promotion__foreach_div, test/inductor/test_foreach.py::ForeachTests::test_type_promotion__foreach_maximum, test/inductor/test_foreach.py::ForeachTests::test_type_promotion__foreach_minimum, test/inductor/test_foreach.py::ForeachTests::test_type_promotion__foreach_mul, test/inductor/test_foreach.py::ForeachTests::test_type_promotion__foreach_sub, test/inductor/test_foreach.py::ForeachTests::test_zero_elems 2024-08-08T21:59:55.4889774Z 2024-08-08T21:59:59.2901623Z Running inductor/test_inductor_freezing 1/1 ... [2024-08-08 21:59:59.289618] 2024-08-08T21:59:59.2906450Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_inductor_freezing.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 21:59:59.290129] 2024-08-08T22:01:13.8390756Z 2024-08-08T22:01:13.8393647Z inductor/test_inductor_freezing 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_inductor_freezing_1.1_e3b22c60bf81e046_.log 2024-08-08T22:01:13.8408914Z Running 22 items in this shard: test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_aliased_param_return_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_autocast_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_conv_bn_with_multi_bn_share_conv_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_conv_functional_bn_with_multi_bn_share_conv_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_conv_layout_convert_with_view_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_conv_multiple_uses_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_conv_weight_layout_convert_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_conv_with_as_strided_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_cpp_wrapper_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_dont_change_dtype_folding_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_error_on_eager_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_folded_conv_bn_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_folded_conv_bn_with_module_sharing_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_folded_conv_functional_bn_with_module_sharing_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_mm_concat_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_mutation_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_param_deallocated_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_redundant_clone_for_layout_convert_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_rng_op_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_symint_not_folded_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_unequal_bias_horizontal_addmm_fusion_cpu, test/inductor/test_inductor_freezing.py::FreezingCpuTests::test_unfolded_bn_cpu 2024-08-08T22:01:13.8421849Z 2024-08-08T22:01:17.7492490Z Running inductor/test_fx_fusion 1/1 ... [2024-08-08 22:01:17.748585] 2024-08-08T22:01:17.7495543Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_fx_fusion.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 22:01:17.749058] 2024-08-08T22:01:23.1340695Z 2024-08-08T22:01:23.1342654Z inductor/test_fx_fusion 1/1 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_fx_fusion_1.1_058dd8a76b9ae7a7_.log 2024-08-08T22:01:23.1347926Z Running 4 items in this shard: test/inductor/test_fx_fusion.py::TestFxFusion::test_linear_permute_fusion, test/inductor/test_fx_fusion.py::TestFxFusion::test_permute_bmm_fusion, test/inductor/test_fx_fusion.py::TestFxFusion::test_permute_linear_fusion, test/inductor/test_fx_fusion.py::TestFxFusion::test_sink_cat_after_pointwise 2024-08-08T22:01:23.1351272Z 2024-08-08T22:01:26.9933710Z Running inductor/test_cpu_cpp_wrapper 1/13 ... [2024-08-08 22:01:26.992815] 2024-08-08T22:01:26.9938173Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_cpu_cpp_wrapper.py', '-m', 'not serial', '--shard-id=1', '--num-shards=13', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 22:01:26.993297] 2024-08-08T22:01:35.0736111Z 2024-08-08T22:01:35.0738601Z inductor/test_cpu_cpp_wrapper 1/13 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_cpu_cpp_wrapper_1.13_70654c14796a0849_.log 2024-08-08T22:01:35.0739885Z 2024-08-08T22:01:38.9157370Z Running inductor/test_cpu_cpp_wrapper 10/13 ... [2024-08-08 22:01:38.915124] 2024-08-08T22:01:38.9161808Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_cpu_cpp_wrapper.py', '-m', 'not serial', '--shard-id=10', '--num-shards=13', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 22:01:38.915651] 2024-08-08T22:01:47.0464795Z 2024-08-08T22:01:47.0466845Z inductor/test_cpu_cpp_wrapper 10/13 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_cpu_cpp_wrapper_10.13_82dc84b58f48cb76_.log 2024-08-08T22:01:47.0468166Z 2024-08-08T22:01:50.9441850Z Running inductor/test_cpu_cpp_wrapper 11/13 ... [2024-08-08 22:01:50.943499] 2024-08-08T22:01:50.9445458Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_cpu_cpp_wrapper.py', '-m', 'not serial', '--shard-id=11', '--num-shards=13', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-08 22:01:50.944004] 2024-08-08T22:01:59.1245611Z 2024-08-08T22:01:59.1248154Z inductor/test_cpu_cpp_wrapper 11/13 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_cpu_cpp_wrapper_11.13_59ec4d53e1b90e6f_.log 2024-08-08T22:01:59.1249651Z 2024-08-08T22:07:00.0824929Z 2024-08-08T22:07:00.0826664Z test_ops_fwd_gradients 2/2 was successful, full logs can be found in artifacts with path test/test-reports/test_ops_fwd_gradients_2.2_f381daefe060f537_.log 2024-08-08T22:07:00.1995766Z Running 1566 items in this shard: test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_H_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_T_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___getitem___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rdiv___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rdiv___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rmatmul___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rmul___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rpow___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rsub___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad___rsub___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__batch_norm_with_update_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__chunk_cat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__chunk_cat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__native_batch_norm_legit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__softmax_backward_data_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__unsafe_masked_index_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__unsafe_masked_index_put_accumulate_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad__upsample_bilinear2d_aa_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_abs_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_acos_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_acosh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_add_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addbmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addcdiv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addcmul_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addcmul_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addmm_decomposed_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addmv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_addr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_all_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_allclose_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_any_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_arange_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_argmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_argwhere_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_as_strided_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_as_strided_partial_views_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_as_strided_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_asin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atanh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atleast_1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atleast_2d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atleast_3d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_atleast_3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_baddbmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_baddbmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_bernoulli_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_bfloat16_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_block_diag_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_bmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_broadcast_tensors_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_broadcast_tensors_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_bucketize_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_byte_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cartesian_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cdist_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cdouble_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ceil_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cfloat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_chalf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_char_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_char_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cholesky_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cholesky_inverse_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cholesky_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_chunk_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_chunk_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_combinations_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_complex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_conj_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_conj_physical_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_conj_physical_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_constant_pad_nd_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_contiguous_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_contiguous_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_corrcoef_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cos_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cos_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cosh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cosh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_count_nonzero_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_count_nonzero_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cummin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cumsum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cumulative_trapezoid_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_cumulative_trapezoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_deg2rad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diag_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diag_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diagflat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diagflat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diagonal_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diagonal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diagonal_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_diff_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_digamma_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_dist_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_div_floor_rounding_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_div_no_rounding_mode_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_div_trunc_rounding_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_dot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_dot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_dstack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_dstack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_einsum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_einsum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_empty_permuted_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_empty_strided_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_eq_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_equal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_erf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_erfinv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_exp2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_exp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_expand_as_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_expand_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_expand_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_expand_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_expm1_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_fft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_fftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_hfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_hfft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_hfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ifft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ifft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ifft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ifft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ifftshift_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ihfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_ihfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_irfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_irfftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_irfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fft_rfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fill_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_flatten_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_flatten_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_flip_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fliplr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_flipud_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_float_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_float_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_float_power_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_float_power_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_floor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_floor_divide_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_fmod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_frac_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_full_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ge_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_geometric_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_geqrf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_geqrf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_gradient_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_gradient_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_grid_sampler_2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_half_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_heaviside_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_hsplit_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_i0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_imag_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_add_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_add_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_index_select_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isfinite_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isnan_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isreal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_isreal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_item_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_item_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_2inputs_2outputs_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_4inputs_with_extra_args_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_binary_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_binary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_jiterator_unary_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_kron_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_kron_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ldexp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_le_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lerp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_cholesky_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_cond_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_cross_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_cross_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_det_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_det_singular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_diagonal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_diagonal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_eig_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_eigvals_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_eigvalsh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_eigvalsh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_householder_product_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_inv_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_ldl_factor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_ldl_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lstsq_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_lu_factor_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_matrix_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_matrix_power_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_matrix_rank_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_matrix_rank_hermitian_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_norm_subgradients_at_zero_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_pinv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_pinv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_pinv_singular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_slogdet_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_solve_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_solve_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_solve_triangular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_tensorinv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_tensorsolve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_tensorsolve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_vander_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_vander_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linalg_vector_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linspace_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linspace_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linspace_tensor_overload_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_linspace_tensor_overload_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log1p_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log_normal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log_softmax_with_dtype_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_log_softmax_with_dtype_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logaddexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logdet_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logical_and_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logical_not_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logical_or_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logical_or_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logical_xor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_logspace_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_long_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lu_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lu_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lu_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_lu_unpack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mH_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mT_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_argmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_cumprod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_cumsum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_fill_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_normalize_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_normalize_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_prod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_softmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_masked_sum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_matrix_exp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_max_binary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_maximum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_meshgrid_list_of_tensors_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_meshgrid_list_of_tensors_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_min_reduction_with_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mode_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_movedim_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mul_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_mvlgamma_mvlgamma_p_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nan_to_num_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nanmean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nanmean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nanmedian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_narrow_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_narrow_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_native_batch_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_native_layer_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ne_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_neg_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_empty_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_empty_strided_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_full_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_full_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_ones_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_ones_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_zeros_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_new_zeros_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_adaptive_avg_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_adaptive_max_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_adaptive_max_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_alpha_dropout_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_batch_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_batch_norm_without_cudnn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_bilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_binary_cross_entropy_with_logits_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv2d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv_transpose2d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_conv_transpose3d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_cosine_embedding_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_cosine_similarity_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_dropout3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_embedding_bag_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_fractional_max_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_glu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_grid_sample_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_group_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_hardshrink_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_hardsigmoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_hardswish_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_huber_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_instance_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_interpolate_area_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_interpolate_nearest_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_interpolate_trilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_kl_div_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_l1_loss_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_layer_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_linear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_logsigmoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_max_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_max_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_max_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_max_unpool1d_grad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_max_unpool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_max_unpool3d_grad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_mish_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_multilabel_soft_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_normalize_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_normalize_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_circular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_constant_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_reflect_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_replicate_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_replicate_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pad_replicate_negative_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pixel_shuffle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_pixel_unshuffle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_relu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_rms_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_scaled_dot_product_attention_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_silu_complex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_silu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_soft_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_softmin_with_dtype_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_softmin_with_dtype_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_softplus_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_softsign_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_tanhshrink_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_threshold_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_triplet_margin_loss_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_triplet_margin_with_distance_loss_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nn_functional_upsample_bilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nonzero_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_nonzero_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_norm_fro_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_norm_inf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_norm_nuc_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_normal_number_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ones_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ormqr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_permute_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_pinverse_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_pinverse_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_polar_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_polygamma_polygamma_n_0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_polygamma_polygamma_n_1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_polygamma_polygamma_n_2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_polygamma_polygamma_n_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_polygamma_polygamma_n_4_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_positive_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_pow_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_pow_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_put_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_qr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_quantile_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_randn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_randn_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_ravel_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_real_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_reciprocal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_remainder_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_repeat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_repeat_interleave_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_resize__cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_resize_as__cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_resize_as__cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_resolve_conj_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_resolve_conj_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_roll_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_rot90_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_round_decimals_0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_round_decimals_neg_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_rsqrt_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_rsqrt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_rsub_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scalar_tensor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scatter_add_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scatter_reduce_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_scatter_reduce_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_searchsorted_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_select_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sgn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_short_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_short_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sigmoid_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sign_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_cosine_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_gaussian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_general_hamming_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_hamming_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_signal_windows_kaiser_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sin_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sinc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_slice_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_slice_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sparse_mm_reduce_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sparse_sampled_addmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_bessel_j0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_chebyshev_polynomial_u_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_chebyshev_polynomial_w_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_erfcx_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_hermite_polynomial_h_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_hermite_polynomial_he_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_i1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_laguerre_polynomial_l_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_log_ndtr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_modified_bessel_i0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_modified_bessel_i1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_polygamma_special_polygamma_n_0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_shifted_chebyshev_polynomial_u_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_special_spherical_bessel_j0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_split_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_split_list_args_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_split_list_args_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_split_with_sizes_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sqrt_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_square_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_squeeze_multiple_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_squeeze_multiple_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_stack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_std_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_std_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_std_mean_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sub_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sum_to_size_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_sum_to_size_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_svd_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_svd_lowrank_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_t_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_t_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_take_along_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_take_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tanh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tensor_split_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tensor_split_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tensordot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tensordot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_tile_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_to_sparse_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_topk_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_trace_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_transpose_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_transpose_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_trapezoid_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_triangular_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_triangular_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_triu_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_trunc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unbind_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unflatten_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unfold_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unfold_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_uniform_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_uniform_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unique_consecutive_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unique_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unsafe_chunk_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unsafe_chunk_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_unsqueeze_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_var_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_var_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_var_mean_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_var_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_vdot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_view_as_complex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_view_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_view_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_view_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_vstack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_where_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_xlogy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_fn_fwgrad_bwgrad_zeros_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_H_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_T_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___getitem___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___getitem___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rdiv___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rdiv___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rmod___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rmul___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD___rsub___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__batch_norm_with_update_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__chunk_cat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__chunk_cat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__native_batch_norm_legit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__softmax_backward_data_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__unsafe_masked_index_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__unsafe_masked_index_put_accumulate_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__unsafe_masked_index_put_accumulate_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD__upsample_bilinear2d_aa_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_abs_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_acos_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_acosh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_acosh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_add_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addbmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addcdiv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addmv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addmv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_addr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_all_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_all_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_allclose_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_aminmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_any_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_as_strided_partial_views_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_as_strided_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_asin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_asinh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atan2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atan_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atan_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atanh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atanh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atleast_1d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atleast_1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atleast_3d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_atleast_3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_baddbmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_bernoulli_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_bfloat16_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_block_diag_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_bmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_bool_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_broadcast_tensors_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_broadcast_tensors_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_broadcast_to_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_broadcast_to_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_byte_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cartesian_prod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cartesian_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cauchy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ceil_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_chalf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_char_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_char_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cholesky_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cholesky_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cholesky_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_chunk_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_clamp_max_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_clamp_min_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_clone_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_column_stack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_combinations_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_conj_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_constant_pad_nd_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_constant_pad_nd_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_contiguous_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_contiguous_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_corrcoef_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cos_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cosh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_count_nonzero_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_count_nonzero_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cross_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cumprod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cumprod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cumsum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_cumsum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_deg2rad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diag_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diag_embed_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diagonal_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diagonal_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diagonal_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_diff_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_dist_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_div_floor_rounding_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_div_no_rounding_mode_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_div_no_rounding_mode_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_div_trunc_rounding_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_dot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_dsplit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_dstack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_einsum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_einsum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_empty_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_empty_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_empty_permuted_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_empty_permuted_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_empty_strided_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_eq_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_equal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_erf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_erfinv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_exp2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_exp2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_expand_as_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_expand_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_expand_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_expm1_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_expm1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_eye_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_eye_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_fft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_fftshift_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_hfft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_hfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_hfft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ifft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ifftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ifftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ifftshift_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ihfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_ihfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_irfft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_irfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_irfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_rfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fft_rfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fill_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_flatten_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_flatten_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_flip_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_flip_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fliplr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_flipud_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_float_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_float_power_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_floor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_fmod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_frexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_full_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_full_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_gather_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_gather_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ge_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_geometric_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_geqrf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_geqrf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_gradient_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_gt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_half_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_half_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_heaviside_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_hsplit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_i0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_igamma_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_imag_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_fill_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_fill_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_put_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_put_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_reduce_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_index_select_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_inner_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_int_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isfinite_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isinf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isposinf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isreal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_isreal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_istft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_item_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_item_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_2inputs_2outputs_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_2inputs_2outputs_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_4inputs_with_extra_args_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_binary_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_jiterator_unary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_kron_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ldexp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_le_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lerp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_cholesky_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_cholesky_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_cond_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_cond_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_det_singular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_diagonal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_diagonal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_eig_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_eigh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_eigvals_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_inv_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_ldl_factor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_ldl_factor_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lstsq_grad_oriented_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lu_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_lu_factor_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_matrix_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_matrix_rank_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_multi_dot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_qr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_slogdet_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_solve_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_solve_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_solve_triangular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_solve_triangular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_svdvals_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_svdvals_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_tensorinv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_tensorsolve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_linalg_vander_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log_normal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log_softmax_with_dtype_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_log_softmax_with_dtype_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logaddexp2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logaddexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logcumsumexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logdet_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logdet_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logical_and_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logical_not_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logical_xor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logspace_tensor_overload_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logspace_tensor_overload_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_logsumexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_long_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_long_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lu_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lu_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_lu_unpack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mH_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_argmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_cumprod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_cumsum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_fill_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_log_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_logaddexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_logsumexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_normalize_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_normalize_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_prod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_std_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_std_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_sum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_masked_var_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_matmul_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_matmul_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_max_pool2d_with_indices_backward_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_max_reduction_no_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_median_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_meshgrid_list_of_tensors_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_meshgrid_variadic_tensors_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_meshgrid_variadic_tensors_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_min_reduction_no_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_movedim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mul_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_mvlgamma_mvlgamma_p_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nan_to_num_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nanmean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nanquantile_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nansum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nansum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_narrow_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_narrow_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_native_batch_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ne_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_neg_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_empty_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_empty_strided_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_full_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_full_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_new_ones_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nextafter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_adaptive_avg_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_adaptive_avg_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_avg_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_batch_norm_without_cudnn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_bilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_binary_cross_entropy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_binary_cross_entropy_with_logits_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_channel_shuffle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv2d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv_transpose1d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv_transpose1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv_transpose2d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_conv_transpose2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_cross_entropy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_ctc_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_elu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_embedding_bag_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_embedding_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_feature_alpha_dropout_with_train_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_feature_alpha_dropout_without_train_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_fractional_max_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_gelu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_glu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_grid_sample_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_group_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_hardshrink_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_hardswish_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_hinge_embedding_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_huber_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_interpolate_area_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_interpolate_bicubic_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_interpolate_bilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_interpolate_linear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_interpolate_trilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_kl_div_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_l1_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_layer_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_leaky_relu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_logsigmoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_margin_ranking_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_max_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_max_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_max_unpool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_max_unpool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_max_unpool2d_grad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_max_unpool3d_grad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_mse_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_multi_head_attention_forward_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_multilabel_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_normalize_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_circular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_constant_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_replicate_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_replicate_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_replicate_negative_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pad_replicate_negative_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pairwise_distance_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pdist_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_pixel_unshuffle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_relu6_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_rms_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_scaled_dot_product_attention_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_silu_complex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_softmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_softshrink_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_softsign_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_softsign_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_threshold_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_triplet_margin_with_distance_loss_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_triplet_margin_with_distance_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nn_functional_upsample_nearest_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_nonzero_static_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_norm_fro_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_norm_inf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_norm_nuc_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_normal_in_place_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_normal_number_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ones_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_outer_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_outer_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_pca_lowrank_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_permute_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_permute_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_pinverse_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_pinverse_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_polygamma_polygamma_n_0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_polygamma_polygamma_n_2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_polygamma_polygamma_n_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_polygamma_polygamma_n_4_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_positive_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_pow_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_prod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_put_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_qr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_rand_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_randint_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_randint_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_randn_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ravel_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_ravel_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_reciprocal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_renorm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_repeat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_reshape_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_reshape_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_resize__cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_resize__cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_resize_as__cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_resolve_conj_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_roll_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_roll_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_rot90_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_round_decimals_0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_round_decimals_neg_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_rsqrt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_rsub_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_rsub_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scalar_tensor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scalar_tensor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scatter_add_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scatter_add_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_searchsorted_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_select_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_select_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sgn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sigmoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_blackman_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_cosine_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_general_hamming_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signal_windows_hamming_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_signbit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_slice_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_softmax_with_dtype_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sort_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sparse_mm_reduce_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sparse_sampled_addmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sparse_sampled_addmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_bessel_j0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_chebyshev_polynomial_v_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_erfcx_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_hermite_polynomial_he_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_i1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_i1e_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_legendre_polynomial_p_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_modified_bessel_i0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_modified_bessel_k1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_ndtr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_polygamma_special_polygamma_n_0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_scaled_modified_bessel_k0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_shifted_chebyshev_polynomial_t_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_shifted_chebyshev_polynomial_w_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_xlog1py_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_special_zeta_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_split_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_split_list_args_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_split_with_sizes_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_split_with_sizes_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_split_with_sizes_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sqrt_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_square_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_squeeze_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_squeeze_multiple_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_stack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_stack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_std_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_std_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_std_mean_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_std_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_stft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sub_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sum_to_size_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_sum_to_size_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_svd_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_svd_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_svd_lowrank_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_svd_lowrank_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_t_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tan_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tanh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tensor_split_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tensordot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tensordot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tile_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_to_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_torch_ops_aten__safe_softmax_default_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_transpose_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_trapezoid_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_trapz_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_triangular_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_tril_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_true_divide_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_true_divide_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unbind_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unflatten_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unfold_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_uniform_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unique_consecutive_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unsafe_chunk_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unsafe_chunk_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unsafe_split_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unsqueeze_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unsqueeze_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_unsqueeze_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_var_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_var_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_var_mean_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_vdot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_vdot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_view_as_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_view_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_view_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_zero__cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_forward_mode_AD_zeros_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___getitem___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___radd___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rdiv___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rmod___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rmul___cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rmul___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD___rpow___cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__batch_norm_with_update_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__chunk_cat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__chunk_cat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__native_batch_norm_legit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__segment_reduce_offsets_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__softmax_backward_data_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__unsafe_masked_index_put_accumulate_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__unsafe_masked_index_put_accumulate_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD__upsample_bilinear2d_aa_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_acos_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_add_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_add_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addcdiv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addcdiv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addcmul_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addmv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addmv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_addr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_alias_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_alias_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_all_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_all_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_allclose_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_angle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_any_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_arange_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_argwhere_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_argwhere_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_as_strided_partial_views_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_as_strided_partial_views_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_as_strided_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_asin_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_asin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_asinh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atan2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atan_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atan_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atanh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atleast_1d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atleast_1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_atleast_3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_baddbmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_bfloat16_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_bfloat16_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_block_diag_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_bool_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_broadcast_tensors_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_broadcast_tensors_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_broadcast_to_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cartesian_prod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cartesian_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cdist_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cdouble_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ceil_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cfloat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_chalf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cholesky_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cholesky_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cholesky_inverse_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cholesky_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_chunk_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_clamp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_clamp_max_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_clamp_min_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_clone_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_column_stack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_column_stack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_combinations_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_constant_pad_nd_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_constant_pad_nd_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_contiguous_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_corrcoef_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_corrcoef_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cos_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cos_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cosh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_count_nonzero_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cov_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cross_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cummax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cummin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cumprod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_cumsum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diag_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diag_embed_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diagflat_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diagonal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diagonal_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diff_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_diff_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_dist_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_div_floor_rounding_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_dot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_dstack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_dstack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_empty_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_empty_strided_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_empty_strided_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_eq_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_equal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_equal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_erf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_erfc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_exp2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_exp_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_expand_as_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_expand_as_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_expand_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_expand_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_exponential_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_fft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_fft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_fftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_fftshift_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_hfft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_hfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_hfft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_hfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_hfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ifft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ifftshift_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ifftshift_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ihfft2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_ihfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_irfft2_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_irfft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_irfftn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_irfftn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fft_rfft_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fill_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_flatten_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_flip_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fliplr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_flipud_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_float_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_float_power_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_float_power_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_floor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_floor_divide_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_fmod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_frexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_full_like_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_full_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_gather_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_gather_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_geometric_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_geqrf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_gradient_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_grid_sampler_2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_hsplit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_igamma_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_imag_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_add_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_add_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_reduce_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_reduce_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_index_select_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_inner_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_inner_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_int_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_int_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isclose_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isfinite_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isfinite_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isinf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isnan_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isneginf_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isreal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_isreal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_istft_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_item_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_2inputs_2outputs_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_2inputs_2outputs_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_4inputs_with_extra_args_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_binary_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_binary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_binary_return_by_ref_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_unary_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_jiterator_unary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_kron_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_kron_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_kthvalue_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ldexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_cholesky_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_cross_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_det_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_det_singular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_det_singular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_eig_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_eig_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_eigh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_eigh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_eigvals_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_eigvalsh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_eigvalsh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_inv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_inv_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_ldl_factor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_ldl_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lstsq_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lu_factor_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_lu_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_matrix_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_matrix_rank_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_multi_dot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_pinv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_pinv_hermitian_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_pinv_hermitian_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_pinv_singular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_qr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_qr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_solve_ex_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_solve_ex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_svdvals_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_svdvals_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_tensorsolve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_vander_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_vecdot_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linalg_vector_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linspace_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linspace_tensor_overload_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_linspace_tensor_overload_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log10_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log1p_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log1p_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_log_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logaddexp2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logaddexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logdet_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logdet_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logical_and_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logical_not_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logical_not_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logical_or_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logical_or_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logical_xor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logical_xor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_logit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_long_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_long_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lu_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lu_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lu_solve_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_lu_unpack_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mH_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mT_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_argmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_cumprod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_cumprod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_fill_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_fill_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_log_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_logaddexp_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_normalize_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_select_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_select_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_softmax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_std_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_sum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_masked_var_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_matmul_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_max_binary_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_max_pool2d_with_indices_backward_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_max_reduction_no_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_max_reduction_with_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_median_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_meshgrid_list_of_tensors_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_meshgrid_list_of_tensors_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_meshgrid_variadic_tensors_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mode_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_movedim_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mul_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_multinomial_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mv_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_mv_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nansum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nansum_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_narrow_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_narrow_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_narrow_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_narrow_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_native_dropout_backward_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ne_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_neg_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_neg_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_empty_strided_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_empty_strided_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_full_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_new_zeros_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nextafter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_adaptive_avg_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_adaptive_max_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_adaptive_max_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_avg_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_bilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_binary_cross_entropy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_binary_cross_entropy_with_logits_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_celu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv1d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv_transpose1d_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_conv_transpose3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_cosine_embedding_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_cosine_similarity_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_cross_entropy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_dropout3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_embedding_bag_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_feature_alpha_dropout_with_train_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_feature_alpha_dropout_without_train_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_fractional_max_pool2d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_gaussian_nll_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_hardshrink_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_hardsigmoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_hardswish_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_hardtanh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_hinge_embedding_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_interpolate_area_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_interpolate_bicubic_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_interpolate_nearest-exact_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_l1_loss_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_layer_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_linear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_local_response_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_margin_ranking_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_max_pool1d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_max_pool3d_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_max_unpool1d_grad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_max_unpool3d_grad_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_mish_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_mse_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_multi_head_attention_forward_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_multi_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_normalize_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_circular_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_circular_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_constant_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_reflect_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_reflect_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_replicate_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pad_replicate_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pairwise_distance_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pixel_shuffle_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_pixel_unshuffle_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_poisson_nll_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_rms_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_selu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_silu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_soft_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_softmin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_softmin_with_dtype_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_softplus_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_softsign_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_tanhshrink_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_triplet_margin_loss_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_triplet_margin_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_triplet_margin_with_distance_loss_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_triplet_margin_with_distance_loss_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_unfold_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_upsample_bilinear_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nn_functional_upsample_nearest_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nonzero_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nonzero_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_nonzero_static_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_norm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_norm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_norm_fro_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_norm_inf_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_norm_nuc_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_norm_nuc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_normal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_normal_in_place_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_normal_in_place_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ones_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ones_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ormqr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_ormqr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_outer_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_pca_lowrank_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_permute_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_pinverse_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_polygamma_polygamma_n_1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_polygamma_polygamma_n_2_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_polygamma_polygamma_n_4_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_pow_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_prod_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_prod_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_put_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_qr_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_rad2deg_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_rand_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_randn_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_randn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_randn_like_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_real_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_real_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_reciprocal_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_reciprocal_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_renorm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_repeat_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_resize_as__cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_round_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_round_decimals_neg_3_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_rsqrt_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_rsub_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scalar_tensor_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scalar_tensor_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scatter_add_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scatter_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scatter_reduce_amax_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_scatter_reduce_amin_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_searchsorted_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_select_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_select_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_select_scatter_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sgn_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sigmoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_cosine_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_exponential_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_general_hamming_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_hann_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signal_windows_kaiser_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_signbit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sinc_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sinh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sinh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_softmax_with_dtype_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sort_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sparse_mm_reduce_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sparse_sampled_addmm_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sparse_sampled_addmm_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_airy_ai_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_bessel_j1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_bessel_y0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_chebyshev_polynomial_t_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_chebyshev_polynomial_w_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_hermite_polynomial_he_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_i0e_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_i1e_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_laguerre_polynomial_l_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_log_ndtr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_modified_bessel_i0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_modified_bessel_k1_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_ndtr_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_scaled_modified_bessel_k0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_shifted_chebyshev_polynomial_t_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_shifted_chebyshev_polynomial_u_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_shifted_chebyshev_polynomial_v_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_shifted_chebyshev_polynomial_w_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_spherical_bessel_j0_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_xlog1py_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_special_zeta_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_split_with_sizes_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_split_with_sizes_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sqrt_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_square_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_square_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_squeeze_multiple_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_stack_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_std_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_std_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_std_mean_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_std_mean_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_std_mean_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sub_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sum_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sum_to_size_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_sum_to_size_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_t_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_t_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_take_along_dim_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_take_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_take_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tanh_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tanh_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tensor_split_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tensor_split_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tile_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_to_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_to_sparse_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_trace_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_trapezoid_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_trapezoid_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_trapz_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_triangular_solve_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_tril_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_triu_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_triu_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_true_divide_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unbind_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unbind_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unflatten_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_uniform_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_uniform_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unique_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unsqueeze_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unsqueeze_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_unsqueeze_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_var_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_var_mean_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_var_unbiased_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_var_unbiased_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_vdot_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_view_as_complex_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_view_as_real_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_view_copy_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_view_copy_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_view_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_view_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_vsplit_cuda_float64, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_where_cuda_complex128, test/test_ops_fwd_gradients.py::TestFwdGradientsCUDA::test_inplace_forward_mode_AD_zeros_like_cuda_float64 2024-08-08T22:07:00.2982528Z 2024-08-08T22:07:01.1277592Z 2024-08-08T22:07:01.1278711Z real 75m13.588s 2024-08-08T22:07:01.1279648Z user 138m4.389s 2024-08-08T22:07:01.1280217Z sys 8m16.220s 2024-08-08T22:07:01.1281069Z + assert_git_not_dirty 2024-08-08T22:07:01.1282809Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build != *rocm* ]] 2024-08-08T22:07:01.1284406Z + [[ linux-focal-cuda12.4-py3.10-gcc9-experimental-split-build != *xla* ]] 2024-08-08T22:07:01.1285784Z ++ git status --porcelain 2024-08-08T22:07:01.1287965Z ++ grep -v '?? third_party' 2024-08-08T22:07:04.1043700Z ++ true 2024-08-08T22:07:04.1044465Z + git_status= 2024-08-08T22:07:04.1045503Z + [[ -n '' ]] 2024-08-08T22:07:04.1046223Z + cleanup_workspace 2024-08-08T22:07:04.1048240Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-08T22:07:04.1050222Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-08T22:07:04.1051434Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-08T22:07:04.1052287Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-08T22:07:04.1053347Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-08T22:07:04.1054485Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-08T22:07:04.1055359Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-08T22:07:04.7700737Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-08T22:07:04.7701421Z cat test/**/*_toprint.log || true 2024-08-08T22:07:04.7712807Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T22:07:04.7713422Z env: 2024-08-08T22:07:04.7713759Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:04.7714324Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:04.7715248Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:04.7716084Z ##[endgroup] 2024-08-08T22:07:04.7807712Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-08T22:07:04.7847138Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-08T22:07:04.7847785Z kill "$MONITOR_SCRIPT_PID" 2024-08-08T22:07:04.7854545Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T22:07:04.7855175Z env: 2024-08-08T22:07:04.7855518Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:04.7856064Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:04.7857007Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:04.7857856Z MONITOR_SCRIPT_PID: 79878 2024-08-08T22:07:04.7858274Z ##[endgroup] 2024-08-08T22:07:04.8105815Z Prepare all required actions 2024-08-08T22:07:04.8106418Z Getting action download info 2024-08-08T22:07:05.0484151Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-08T22:07:05.2181790Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-08T22:07:05.2182368Z with: 2024-08-08T22:07:05.2183000Z file-suffix: test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287 2024-08-08T22:07:05.2183816Z s3-bucket: gha-artifacts 2024-08-08T22:07:05.2184217Z env: 2024-08-08T22:07:05.2184555Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:05.2185111Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:05.2186034Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:05.2186856Z ##[endgroup] 2024-08-08T22:07:05.2219279Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-08T22:07:05.2220121Z # Remove any previous test jsons if they exist 2024-08-08T22:07:05.2220744Z rm -f test-jsons-*.zip 2024-08-08T22:07:05.2221390Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-08T22:07:05.2231042Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T22:07:05.2231651Z env: 2024-08-08T22:07:05.2231987Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:05.2232546Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:05.2233709Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:05.2234869Z FILE_SUFFIX: test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287 2024-08-08T22:07:05.2235639Z ##[endgroup] 2024-08-08T22:07:05.2606441Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-08T22:07:05.2641360Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-08T22:07:05.2642203Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-08T22:07:05.2649513Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-08T22:07:05.2653691Z adding: test/test-reports/td_exclusions-b490e2c9fd9035fdbf5b.json (deflated 81%) 2024-08-08T22:07:05.2654695Z adding: test/.pytorch-slow-tests.json (deflated 67%) 2024-08-08T22:07:05.2670972Z adding: test/.pytorch-disabled-tests.json (deflated 89%) 2024-08-08T22:07:05.2705196Z ##[group]Run # Remove any previous test reports if they exist 2024-08-08T22:07:05.2706022Z # Remove any previous test reports if they exist 2024-08-08T22:07:05.2706675Z rm -f test-reports-*.zip 2024-08-08T22:07:05.2707409Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-08T22:07:05.2714345Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T22:07:05.2714972Z env: 2024-08-08T22:07:05.2715311Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:05.2715855Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:05.2716808Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:05.2717921Z FILE_SUFFIX: test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287 2024-08-08T22:07:05.2718670Z ##[endgroup] 2024-08-08T22:07:05.2951104Z adding: test/test-reports/python-pytest/test_decomp/test_decomp-2ee3e81816d96e72.xml (deflated 28%) 2024-08-08T22:07:05.2969905Z adding: test/test-reports/python-pytest/test_decomp/test_decomp-86ac3fdb3ab1fe1f.xml (deflated 92%) 2024-08-08T22:07:05.2971464Z adding: test/test-reports/python-pytest/test_decomp/test_decomp-8c4eac52d80c4fc3.xml (deflated 34%) 2024-08-08T22:07:05.2976264Z adding: test/test-reports/python-pytest/test_decomp/test_decomp-32ed1af9f4a2f5b1.xml (deflated 92%) 2024-08-08T22:07:05.2978518Z adding: test/test-reports/python-pytest/test_model_exports_to_core_aten/test_model_exports_to_core_aten-229d561ba13ba129.xml (deflated 28%) 2024-08-08T22:07:05.2980791Z adding: test/test-reports/python-pytest/test_model_exports_to_core_aten/test_model_exports_to_core_aten-56f959c2fc4a8f34.xml (deflated 58%) 2024-08-08T22:07:05.2982651Z adding: test/test-reports/python-pytest/test_transformers/test_transformers-199b9150371d1e70.xml (deflated 27%) 2024-08-08T22:07:05.3953908Z adding: test/test-reports/python-pytest/test_transformers/test_transformers-fc7fcbdc7cb25791.xml (deflated 99%) 2024-08-08T22:07:05.3955646Z adding: test/test-reports/python-pytest/test_transformers/test_transformers-1f9fa027e9315f70.xml (deflated 36%) 2024-08-08T22:07:05.4098853Z adding: test/test-reports/python-pytest/test_transformers/test_transformers-6f63f59cbd7291bb.xml (deflated 98%) 2024-08-08T22:07:05.4102597Z adding: test/test-reports/python-pytest/inductor.test_torchinductor_opinfo/inductor.test_torchinductor_opinfo-fae65b250c792f6e.xml (deflated 28%) 2024-08-08T22:07:05.4106646Z adding: test/test-reports/python-pytest/inductor.test_torchinductor_opinfo/inductor.test_torchinductor_opinfo-8756fe5fa793242f.xml (deflated 28%) 2024-08-08T22:07:05.4108817Z adding: test/test-reports/python-pytest/inductor.test_torchinductor_opinfo/inductor.test_torchinductor_opinfo-8f84597b8b8d0326.xml (deflated 28%) 2024-08-08T22:07:05.4111072Z adding: test/test-reports/python-pytest/inductor.test_torchinductor_opinfo/inductor.test_torchinductor_opinfo-b2a520f487ec2623.xml (deflated 27%) 2024-08-08T22:07:05.4113976Z adding: test/test-reports/python-pytest/inductor.test_torchinductor_opinfo/inductor.test_torchinductor_opinfo-9d5d0ada4f650d81.xml (deflated 97%) 2024-08-08T22:07:05.4116878Z adding: test/test-reports/python-pytest/inductor.test_torchinductor_opinfo/inductor.test_torchinductor_opinfo-48e23559a8424c5e.xml (deflated 97%) 2024-08-08T22:07:05.4119930Z adding: test/test-reports/python-pytest/inductor.test_torchinductor_opinfo/inductor.test_torchinductor_opinfo-e1427ce249a10c1c.xml (deflated 97%) 2024-08-08T22:07:05.4125838Z adding: test/test-reports/python-pytest/inductor.test_torchinductor_opinfo/inductor.test_torchinductor_opinfo-fa70e1303d5730b4.xml (deflated 97%) 2024-08-08T22:07:05.4127795Z adding: test/test-reports/python-pytest/functorch.test_ops/functorch.test_ops-9cff26491f999fea.xml (deflated 27%) 2024-08-08T22:07:05.4177164Z adding: test/test-reports/python-pytest/functorch.test_ops/functorch.test_ops-59bd621a12c75a8c.xml (deflated 93%) 2024-08-08T22:07:05.4179051Z adding: test/test-reports/python-pytest/dynamo.test_dynamic_shapes/dynamo.test_dynamic_shapes-e33935be79a269ba.xml (deflated 28%) 2024-08-08T22:07:05.4180962Z adding: test/test-reports/python-pytest/dynamo.test_dynamic_shapes/dynamo.test_dynamic_shapes-ccc0e0406c1d234a.xml (deflated 28%) 2024-08-08T22:07:05.4230285Z adding: test/test-reports/python-pytest/dynamo.test_dynamic_shapes/dynamo.test_dynamic_shapes-8267450c2c92c409.xml (deflated 90%) 2024-08-08T22:07:05.4275569Z adding: test/test-reports/python-pytest/dynamo.test_dynamic_shapes/dynamo.test_dynamic_shapes-284c53f11244cb20.xml (deflated 91%) 2024-08-08T22:07:05.4277272Z adding: test/test-reports/python-pytest/test_ops/test_ops-449f8397106f9c51.xml (deflated 28%) 2024-08-08T22:07:05.4536112Z adding: test/test-reports/python-pytest/test_ops/test_ops-edf71dac88f824e8.xml (deflated 97%) 2024-08-08T22:07:05.4537610Z adding: test/test-reports/python-pytest/test_ops_jit/test_ops_jit-112f209c936b967f.xml (deflated 28%) 2024-08-08T22:07:05.4550849Z adding: test/test-reports/python-pytest/test_ops_jit/test_ops_jit-b966704d64c1252e.xml (deflated 93%) 2024-08-08T22:07:05.4553038Z adding: test/test-reports/python-pytest/test_mkldnn_verbose/test_mkldnn_verbose-c960ea5aad001556.xml (deflated 28%) 2024-08-08T22:07:05.4554751Z adding: test/test-reports/python-pytest/test_mkldnn_verbose/test_mkldnn_verbose-63bbe1124c90bdf0.xml (deflated 50%) 2024-08-08T22:07:05.4557720Z adding: test/test-reports/python-pytest/test_expanded_weights/test_expanded_weights-bd1ccb39c06b5a30.xml (deflated 28%) 2024-08-08T22:07:05.4559989Z adding: test/test-reports/python-pytest/test_expanded_weights/test_expanded_weights-62ed961329f2b8e9.xml (deflated 94%) 2024-08-08T22:07:05.4562014Z adding: test/test-reports/python-pytest/test_ops_fwd_gradients/test_ops_fwd_gradients-7affb67d5a846769.xml (deflated 28%) 2024-08-08T22:07:05.4563848Z adding: test/test-reports/python-pytest/test_ops_fwd_gradients/test_ops_fwd_gradients-b7031ddbb6645e13.xml (deflated 28%) 2024-08-08T22:07:05.4597334Z adding: test/test-reports/python-pytest/test_ops_fwd_gradients/test_ops_fwd_gradients-63b39cfd8468e52e.xml (deflated 95%) 2024-08-08T22:07:05.4636240Z adding: test/test-reports/python-pytest/test_ops_fwd_gradients/test_ops_fwd_gradients-bfc60ba5ecc3e7a0.xml (deflated 95%) 2024-08-08T22:07:05.4639401Z adding: test/test-reports/python-pytest/inductor.test_indexing/inductor.test_indexing-4f5c5e80efd8f47f.xml (deflated 28%) 2024-08-08T22:07:05.4641220Z adding: test/test-reports/python-pytest/inductor.test_indexing/inductor.test_indexing-c213f8b3f98fbe9e.xml (deflated 80%) 2024-08-08T22:07:05.4643955Z adding: test/test-reports/python-pytest/dynamo.test_nops/dynamo.test_nops-a2be09441f1b7e2e.xml (deflated 28%) 2024-08-08T22:07:05.4646938Z adding: test/test-reports/python-pytest/dynamo.test_nops/dynamo.test_nops-351a12fb412c855a.xml (deflated 61%) 2024-08-08T22:07:05.4648706Z adding: test/test-reports/python-pytest/inductor.test_foreach/inductor.test_foreach-e92ba73d5ce11428.xml (deflated 28%) 2024-08-08T22:07:05.4650512Z adding: test/test-reports/python-pytest/inductor.test_foreach/inductor.test_foreach-6b45f534906df2d0.xml (deflated 97%) 2024-08-08T22:07:05.4652712Z adding: test/test-reports/python-pytest/inductor.test_inductor_freezing/inductor.test_inductor_freezing-41c19dafe6b8971d.xml (deflated 28%) 2024-08-08T22:07:05.4654740Z adding: test/test-reports/python-pytest/inductor.test_inductor_freezing/inductor.test_inductor_freezing-9b29fe342933ff1c.xml (deflated 88%) 2024-08-08T22:07:05.4656671Z adding: test/test-reports/python-pytest/inductor.test_fx_fusion/inductor.test_fx_fusion-6fa54a867926ebc0.xml (deflated 28%) 2024-08-08T22:07:05.4658934Z adding: test/test-reports/python-pytest/inductor.test_fx_fusion/inductor.test_fx_fusion-d1a49c42e88e4207.xml (deflated 63%) 2024-08-08T22:07:05.4689377Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-08T22:07:05.4690165Z # Remove any previous usage logs if they exist 2024-08-08T22:07:05.4690790Z rm -f logs-*.zip 2024-08-08T22:07:05.4691618Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-08T22:07:05.4692617Z # so check to see if the file exists first 2024-08-08T22:07:05.4693219Z if [ -f 'usage_log.txt' ]; then 2024-08-08T22:07:05.4693873Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-08T22:07:05.4694473Z fi 2024-08-08T22:07:05.4694894Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-08T22:07:05.4695606Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-08T22:07:05.4696215Z fi 2024-08-08T22:07:05.4703056Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T22:07:05.4703685Z env: 2024-08-08T22:07:05.4704024Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:05.4704569Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:05.4706207Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:05.4707400Z FILE_SUFFIX: test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287 2024-08-08T22:07:05.4708304Z ##[endgroup] 2024-08-08T22:07:05.4808699Z adding: usage_log.txt (deflated 92%) 2024-08-08T22:07:05.5069086Z adding: test/test-reports/test_decomp_1.7_9f0c7d6a7abe2ca9_.log (deflated 48%) 2024-08-08T22:07:05.5072053Z adding: test/test-reports/test_model_exports_to_core_aten_1.1_b61af7ea6bd55146_.log (deflated 51%) 2024-08-08T22:07:05.5075007Z adding: test/test-reports/test_transformers_1.1_121b2bca58ee998a_.log (deflated 50%) 2024-08-08T22:07:05.5078100Z adding: test/test-reports/inductor.test_torchinductor_opinfo_1.18_44931cefcbd141ce_.log (deflated 52%) 2024-08-08T22:07:05.5081732Z adding: test/test-reports/inductor.test_torchinductor_opinfo_6.18_7338181b005d0439_.log (deflated 52%) 2024-08-08T22:07:05.5084395Z adding: test/test-reports/inductor.test_torchinductor_opinfo_11.18_e6b8445cf0ea2a95_.log (deflated 52%) 2024-08-08T22:07:05.5085938Z adding: test/test-reports/inductor.test_torchinductor_opinfo_16.18_d951f924edba608b_.log (deflated 52%) 2024-08-08T22:07:05.5087316Z adding: test/test-reports/functorch.test_ops_4.5_b1198d87da135905_.log (deflated 49%) 2024-08-08T22:07:05.5088639Z adding: test/test-reports/dynamo.test_dynamic_shapes_1.2_d4b209d5fa610097_.log (deflated 51%) 2024-08-08T22:07:05.5089969Z adding: test/test-reports/dynamo.test_dynamic_shapes_2.2_59d31e2422d6ee5f_.log (deflated 51%) 2024-08-08T22:07:05.5091191Z adding: test/test-reports/test_ops_4.4_7b3eb80f9d6251e7_.log (deflated 49%) 2024-08-08T22:07:05.5092322Z adding: test/test-reports/test_ops_jit_2.2_ac3b1d7ee1e84871_.log (deflated 49%) 2024-08-08T22:07:05.5093553Z adding: test/test-reports/inductor.test_aot_inductor_4.9_3a9348364ab7ccbd_.log (stored 0%) 2024-08-08T22:07:05.5094831Z adding: test/test-reports/test_mkldnn_verbose_1.1_65577b130c1a3f65_.log (deflated 50%) 2024-08-08T22:07:05.5096092Z adding: test/test-reports/test_expanded_weights_1.1_168622ef72da6c7d_.log (deflated 50%) 2024-08-08T22:07:05.5097379Z adding: test/test-reports/test_ops_fwd_gradients_1.2_3ce36d6129ae49d5_.log (deflated 50%) 2024-08-08T22:07:05.5098853Z adding: test/test-reports/test_ops_fwd_gradients_2.2_7fdf1936b4363764_.log (deflated 50%) 2024-08-08T22:07:05.5100171Z adding: test/test-reports/inductor.test_cuda_cpp_wrapper_2.3_06f8e2cded401208_.log (stored 0%) 2024-08-08T22:07:05.5101509Z adding: test/test-reports/inductor.test_indexing_1.1_ca45254af25dbed8_.log (deflated 50%) 2024-08-08T22:07:05.5102753Z adding: test/test-reports/dynamo.test_nops_1.1_380789cafc33d817_.log (deflated 50%) 2024-08-08T22:07:05.5104066Z adding: test/test-reports/inductor.test_triton_heuristics_1.1_ed49934f6ed4d398_.log (stored 0%) 2024-08-08T22:07:05.5105394Z adding: test/test-reports/inductor.test_foreach_1.1_7344e9d442d5990d_.log (deflated 50%) 2024-08-08T22:07:05.5106734Z adding: test/test-reports/inductor.test_inductor_freezing_1.1_a96117fa233b2d5c_.log (deflated 52%) 2024-08-08T22:07:05.5108104Z adding: test/test-reports/inductor.test_fx_fusion_1.1_59bf46ab3d7b2bdb_.log (deflated 50%) 2024-08-08T22:07:05.5109456Z adding: test/test-reports/inductor.test_cpu_cpp_wrapper_1.13_d16863c939a3ed8f_.log (stored 0%) 2024-08-08T22:07:05.5110840Z adding: test/test-reports/inductor.test_cpu_cpp_wrapper_10.13_0f38537f8281c893_.log (stored 0%) 2024-08-08T22:07:05.5112194Z adding: test/test-reports/inductor.test_cpu_cpp_wrapper_11.13_5419d3188ef78d34_.log (stored 0%) 2024-08-08T22:07:05.5113579Z adding: test/test-reports/test_model_exports_to_core_aten_1.1_da3fe7c3c7d7e941_.log (deflated 52%) 2024-08-08T22:07:05.6591441Z adding: test/test-reports/test_transformers_1.1_44c310efaea94729_.log (deflated 99%) 2024-08-08T22:07:05.6598303Z adding: test/test-reports/inductor.test_torchinductor_opinfo_1.18_5259973fbe565482_.log (deflated 93%) 2024-08-08T22:07:05.6605114Z adding: test/test-reports/inductor.test_torchinductor_opinfo_6.18_af310ee80f22e4d4_.log (deflated 93%) 2024-08-08T22:07:05.6611913Z adding: test/test-reports/inductor.test_torchinductor_opinfo_11.18_6141d21fc632f41d_.log (deflated 93%) 2024-08-08T22:07:05.6618408Z adding: test/test-reports/inductor.test_torchinductor_opinfo_16.18_36dccc83db37efd3_.log (deflated 93%) 2024-08-08T22:07:05.6680265Z adding: test/test-reports/functorch.test_ops_4.5_a1830a078216e27e_.log (deflated 92%) 2024-08-08T22:07:05.6712382Z adding: test/test-reports/dynamo.test_dynamic_shapes_1.2_ef2ced4b5d4bbf1c_.log (deflated 91%) 2024-08-08T22:07:05.6785168Z adding: test/test-reports/dynamo.test_dynamic_shapes_2.2_c95727dd111c647a_.log (deflated 94%) 2024-08-08T22:07:05.6822169Z adding: test/test-reports/test_decomp_1.7_e987e43d390a110b_.log (deflated 91%) 2024-08-08T22:07:05.6840416Z adding: test/test-reports/test_ops_jit_2.2_50eb32c2d3d3bd56_.log (deflated 91%) 2024-08-08T22:07:05.6841691Z adding: test/test-reports/inductor.test_aot_inductor_4.9_c757054a6ba12aa7_.log (stored 0%) 2024-08-08T22:07:05.6842962Z adding: test/test-reports/test_mkldnn_verbose_1.1_7fa10cd3eaf682b7_.log (deflated 55%) 2024-08-08T22:07:05.6848349Z adding: test/test-reports/test_expanded_weights_1.1_42e061686ac92257_.log (deflated 93%) 2024-08-08T22:07:05.7070287Z adding: test/test-reports/test_ops_4.4_830c00724824914e_.log (deflated 93%) 2024-08-08T22:07:05.7118428Z adding: test/test-reports/test_ops_fwd_gradients_1.2_5879d65d13635670_.log (deflated 94%) 2024-08-08T22:07:05.7119768Z adding: test/test-reports/inductor.test_cuda_cpp_wrapper_2.3_89baaa21f06d7d6d_.log (stored 0%) 2024-08-08T22:07:05.7121068Z adding: test/test-reports/inductor.test_indexing_1.1_67c7a861bad10a91_.log (deflated 77%) 2024-08-08T22:07:05.7122352Z adding: test/test-reports/dynamo.test_nops_1.1_bae74dff6d5edb0e_.log (deflated 60%) 2024-08-08T22:07:05.7123653Z adding: test/test-reports/inductor.test_triton_heuristics_1.1_b514c5a23e73b968_.log (stored 0%) 2024-08-08T22:07:05.7126351Z adding: test/test-reports/inductor.test_foreach_1.1_7fa09e884456c483_.log (deflated 93%) 2024-08-08T22:07:05.7127741Z adding: test/test-reports/inductor.test_inductor_freezing_1.1_e3b22c60bf81e046_.log (deflated 81%) 2024-08-08T22:07:05.7129272Z adding: test/test-reports/inductor.test_fx_fusion_1.1_058dd8a76b9ae7a7_.log (deflated 63%) 2024-08-08T22:07:05.7130720Z adding: test/test-reports/inductor.test_cpu_cpp_wrapper_1.13_70654c14796a0849_.log (stored 0%) 2024-08-08T22:07:05.7132095Z adding: test/test-reports/inductor.test_cpu_cpp_wrapper_10.13_82dc84b58f48cb76_.log (stored 0%) 2024-08-08T22:07:05.7133479Z adding: test/test-reports/inductor.test_cpu_cpp_wrapper_11.13_59ec4d53e1b90e6f_.log (stored 0%) 2024-08-08T22:07:05.7177682Z adding: test/test-reports/test_ops_fwd_gradients_2.2_f381daefe060f537_.log (deflated 94%) 2024-08-08T22:07:05.7214728Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-08T22:07:05.7215633Z # Remove any previous debugging artifacts if they exist 2024-08-08T22:07:05.7216317Z rm -f debug-*.zip 2024-08-08T22:07:05.7216787Z if [ -d 'test/debug' ]; then 2024-08-08T22:07:05.7217393Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-08T22:07:05.7217991Z fi 2024-08-08T22:07:05.7226017Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T22:07:05.7226624Z env: 2024-08-08T22:07:05.7226964Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:05.7227525Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:05.7228444Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:05.7229558Z FILE_SUFFIX: test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287 2024-08-08T22:07:05.7230334Z ##[endgroup] 2024-08-08T22:07:05.7338268Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-08T22:07:05.7338809Z with: 2024-08-08T22:07:05.7339151Z s3-bucket: gha-artifacts 2024-08-08T22:07:05.7339692Z s3-prefix: pytorch/pytorch/10308807531/1/artifact 2024-08-08T22:07:05.7340260Z retention-days: 14 2024-08-08T22:07:05.7340671Z if-no-files-found: warn 2024-08-08T22:07:05.7341110Z path: test-jsons-*.zip 2024-08-08T22:07:05.7341511Z name: artifact 2024-08-08T22:07:05.7341889Z region: us-east-1 2024-08-08T22:07:05.7342262Z env: 2024-08-08T22:07:05.7342574Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:05.7343131Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:05.7344066Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:05.7344868Z ##[endgroup] 2024-08-08T22:07:06.1910676Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-08T22:07:06.1911898Z With the provided path, there will be 1 file uploaded 2024-08-08T22:07:06.1913041Z Uploading to s3 prefix: pytorch/pytorch/10308807531/1/artifact 2024-08-08T22:07:06.4078417Z Starting upload of test-jsons-test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287.zip 2024-08-08T22:07:06.5548787Z Finished upload of test-jsons-test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287.zip 2024-08-08T22:07:06.6764526Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-08T22:07:06.6765073Z with: 2024-08-08T22:07:06.6765431Z s3-bucket: gha-artifacts 2024-08-08T22:07:06.6765983Z s3-prefix: pytorch/pytorch/10308807531/1/artifact 2024-08-08T22:07:06.6766552Z retention-days: 14 2024-08-08T22:07:06.6766962Z if-no-files-found: error 2024-08-08T22:07:06.6767412Z path: test-reports-*.zip 2024-08-08T22:07:06.6767827Z name: artifact 2024-08-08T22:07:06.6768199Z region: us-east-1 2024-08-08T22:07:06.6768565Z env: 2024-08-08T22:07:06.6768879Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:06.6769434Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:06.6770388Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:06.6771191Z ##[endgroup] 2024-08-08T22:07:07.1421357Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-08T22:07:07.1422346Z With the provided path, there will be 1 file uploaded 2024-08-08T22:07:07.1423086Z Uploading to s3 prefix: pytorch/pytorch/10308807531/1/artifact 2024-08-08T22:07:07.1477312Z Starting upload of test-reports-test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287.zip 2024-08-08T22:07:07.4919848Z Finished upload of test-reports-test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287.zip 2024-08-08T22:07:07.5132770Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-08T22:07:07.5133309Z with: 2024-08-08T22:07:07.5133660Z s3-bucket: gha-artifacts 2024-08-08T22:07:07.5134200Z s3-prefix: pytorch/pytorch/10308807531/1/artifact 2024-08-08T22:07:07.5134834Z retention-days: 14 2024-08-08T22:07:07.5135254Z if-no-files-found: ignore 2024-08-08T22:07:07.5135696Z path: logs-*.zip 2024-08-08T22:07:07.5136080Z name: artifact 2024-08-08T22:07:07.5136459Z region: us-east-1 2024-08-08T22:07:07.5136822Z env: 2024-08-08T22:07:07.5137138Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:07.5137694Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:07.5138628Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:07.5139430Z ##[endgroup] 2024-08-08T22:07:07.9762291Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-08T22:07:07.9763654Z With the provided path, there will be 1 file uploaded 2024-08-08T22:07:07.9764387Z Uploading to s3 prefix: pytorch/pytorch/10308807531/1/artifact 2024-08-08T22:07:07.9816284Z Starting upload of logs-test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287.zip 2024-08-08T22:07:08.1254131Z Finished upload of logs-test-default-4-5-amz2023.linux.4xlarge.nvidia.gpu_28538244287.zip 2024-08-08T22:07:08.1466992Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-08T22:07:08.1467526Z with: 2024-08-08T22:07:08.1467887Z s3-bucket: gha-artifacts 2024-08-08T22:07:08.1468430Z s3-prefix: pytorch/pytorch/10308807531/1/artifact 2024-08-08T22:07:08.1468992Z retention-days: 14 2024-08-08T22:07:08.1469405Z if-no-files-found: ignore 2024-08-08T22:07:08.1469846Z path: debug-*.zip 2024-08-08T22:07:08.1470208Z name: artifact 2024-08-08T22:07:08.1470574Z region: us-east-1 2024-08-08T22:07:08.1470940Z env: 2024-08-08T22:07:08.1471255Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:08.1471824Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:08.1472756Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:08.1473555Z ##[endgroup] 2024-08-08T22:07:08.6003947Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-08T22:07:08.6228150Z ##[group]Run # shellcheck disable=SC2156 2024-08-08T22:07:08.6228784Z # shellcheck disable=SC2156 2024-08-08T22:07:08.6229773Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-08T22:07:08.6237352Z shell: /usr/bin/bash -e {0} 2024-08-08T22:07:08.6237785Z env: 2024-08-08T22:07:08.6238113Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:08.6238647Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:08.6239584Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:08.6240418Z ##[endgroup] 2024-08-08T22:07:08.9244899Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-08T22:07:08.9245635Z with: 2024-08-08T22:07:08.9245951Z env: 2024-08-08T22:07:08.9246280Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:08.9246820Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:08.9247754Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:08.9248573Z ##[endgroup] 2024-08-08T22:07:08.9273275Z ##[group]Run set -eou pipefail 2024-08-08T22:07:08.9273788Z set -eou pipefail 2024-08-08T22:07:08.9274224Z  2024-08-08T22:07:08.9274858Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-08T22:07:08.9275648Z for _ in $(seq 1440); do 2024-08-08T22:07:08.9276238Z  # Break if no ssh session exists anymore 2024-08-08T22:07:08.9276858Z  if [ "$(who)" = "" ]; then 2024-08-08T22:07:08.9277594Z  break 2024-08-08T22:07:08.9278044Z  fi 2024-08-08T22:07:08.9278399Z  echo "." 2024-08-08T22:07:08.9278798Z  sleep 5 2024-08-08T22:07:08.9279186Z done 2024-08-08T22:07:08.9286308Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T22:07:08.9286938Z env: 2024-08-08T22:07:08.9287275Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:08.9287820Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:08.9288762Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:08.9289587Z ##[endgroup] 2024-08-08T22:07:08.9320198Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-08T22:07:08.9379698Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-08T22:07:08.9380664Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-08T22:07:08.9381401Z # shellcheck disable=SC2046 2024-08-08T22:07:08.9381997Z docker stop $(docker ps -q) || true 2024-08-08T22:07:08.9382577Z # Prune all of the docker images 2024-08-08T22:07:08.9383138Z docker system prune -af 2024-08-08T22:07:08.9390186Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T22:07:08.9390792Z env: 2024-08-08T22:07:08.9391128Z GIT_DEFAULT_BRANCH: main 2024-08-08T22:07:08.9391692Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-08T22:07:08.9392612Z DOCKER_CONTAINER_ID: eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:08.9393436Z ##[endgroup] 2024-08-08T22:07:09.5242786Z eca2856b64b6 2024-08-08T22:07:10.1339458Z Deleted Containers: 2024-08-08T22:07:10.1340642Z eca2856b64b6868c1b07c5990fa11edf6a1eb2d5ce4a941f9d1af926640b20f3 2024-08-08T22:07:10.1341229Z 2024-08-08T22:07:16.0350696Z Deleted Images: 2024-08-08T22:07:16.0353395Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-08T22:07:16.0357070Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9@sha256:a898a1b237f3fdd8f48bb276749a44016150d2c25ae891805a3c665b34fc8003 2024-08-08T22:07:16.0359739Z deleted: sha256:cd235e6b1be7a0d3099ad6f7c05391c21669cd859ddf645fc846603ac8b56d9d 2024-08-08T22:07:16.0361431Z deleted: sha256:45e0976f575503941a10b3cba4a3ff590bfc203b4802c6de8c59b369d656724a 2024-08-08T22:07:16.0363226Z deleted: sha256:c61acbbfdbad22e2c88d71ca0d29b93d1fcce66c2ae345c9bb96a7dec302c41c 2024-08-08T22:07:16.0364300Z deleted: sha256:ff8a69cb9279e8a03a7762132f9250853a69284719cb945f8a35243187f47279 2024-08-08T22:07:16.0365379Z deleted: sha256:bfa68c5e2a3a3ede4c060e1f6d0d81ab96d1b00799acfae1f56f70b0568711c1 2024-08-08T22:07:16.0366454Z deleted: sha256:34d749c950e5c984c12e879e402a48e25d5b9679a2f76e39d1918cb44ae6a71a 2024-08-08T22:07:16.0367481Z deleted: sha256:d0e25776a85880ce3302ae6db212459666aab48f1334835d87f21de012907a02 2024-08-08T22:07:16.0368886Z deleted: sha256:56f4028ec8948e5bbe1fc78bb1d9be7b0d23ca11d3104bde7f0282fc20e1c127 2024-08-08T22:07:16.0369999Z deleted: sha256:0c794ed2194f5103e859213866ed9909f09810fbf10ce2dbc14c002c01e69f93 2024-08-08T22:07:16.0371092Z deleted: sha256:8402d1b515f17b0dce83caf95ffcb31956a9072bbfb00fb391c2fc2e898ba254 2024-08-08T22:07:16.0372197Z deleted: sha256:c5d33c900427aa286d93c976cf12d3cebdb2768378fff3885020764d25f3bc88 2024-08-08T22:07:16.0374126Z deleted: sha256:604d9b0bd0bbdd7a26f27307b4d8992f6d3b3adebc46b238527cf5f0bedef4c4 2024-08-08T22:07:16.0375873Z deleted: sha256:0cd1ea2753107d557974d29a4bb4730b677ca5573deaf385aec686604c46401f 2024-08-08T22:07:16.0376941Z deleted: sha256:0bac64609c89c877da39f47fd39acf4fa56f1f2a27a08bb46139e425d304ab60 2024-08-08T22:07:16.0378022Z deleted: sha256:4986487e5ed4b36a77cf0eeefc92c8905a6bace35b800b029313648631b6d03d 2024-08-08T22:07:16.0379108Z deleted: sha256:cc720cee203d1b11d91181db2cfb5107dd19bce5e6861f0ef6960c35a25be595 2024-08-08T22:07:16.0380401Z deleted: sha256:7f43efadde3b97ee2c002bbb560a974ba5f46dcccbba24e64163b8616ec796dd 2024-08-08T22:07:16.0381622Z deleted: sha256:ab135f617807e146f11acea7262e2b7608cde27b65c17e3cdde67ccea6b14992 2024-08-08T22:07:16.0382835Z deleted: sha256:5a989dc79511366cc92333e53994e4436738aa240bc3c47e3fbd56730675a2d8 2024-08-08T22:07:16.0383870Z deleted: sha256:a1df2af593e5055f5e2b08c143428588b2185c33be2e8fd9ffbf6550ffb87444 2024-08-08T22:07:16.0384978Z deleted: sha256:ca27bfb2f8cc8d54ae99babc01c9ed877b0f4f6e584a4f02e8b9e5d1d9c87180 2024-08-08T22:07:16.0386026Z deleted: sha256:228989acb158238200f18b4215b8b197b898b7a096119505fc0d3c102f50deee 2024-08-08T22:07:16.0387063Z deleted: sha256:1a9ef9dd08df2580c9727a19d02612712f4423d670cf3ef1987174a1b68614d5 2024-08-08T22:07:16.0388197Z deleted: sha256:20b511fb91716943dad5eeeae90ece21a8df93934737eb36de2c835a9d737ba2 2024-08-08T22:07:16.0389304Z deleted: sha256:8b7868b74d782bb44566be626004b12d6b7512a67161e2ef871a10110522ebb6 2024-08-08T22:07:16.0390242Z deleted: sha256:3a3302a77096f6f56e6f0fc8c8204f83c77da624a0a796a3535ee2e90027f96c 2024-08-08T22:07:16.0391294Z deleted: sha256:6d4f4af85672bbb276ce5e0907b4cfda1bfa97dfa165024fc074a62e4cc71879 2024-08-08T22:07:16.0392382Z deleted: sha256:37e2bcfdc59452203f4a07dac6a0698704c04bbfccbec59e231636aa47af55a3 2024-08-08T22:07:16.0393453Z deleted: sha256:dd486adc1f62b525884ef12618694faf9882f1886bf74146a1ad1e6e22807ef2 2024-08-08T22:07:16.0394551Z deleted: sha256:b270237b781e3f5f003e295c5a93cf7c773a2243a3fb7eb12269d33347136d3a 2024-08-08T22:07:16.0395593Z deleted: sha256:804b5c4dd7aad2c2fca3065276469b495240ab6b7e945e764fe8fdd14115dc86 2024-08-08T22:07:16.0396676Z deleted: sha256:904f81e9115e75fe6eebcbde4cc9ad7d882fc474e265c50e003a3d292007ff0e 2024-08-08T22:07:16.0397741Z deleted: sha256:29a7b758345afa399031b70583f145466e34ab5bfceb8f4e99d39db53ca3507c 2024-08-08T22:07:16.0398780Z deleted: sha256:f142ce97b4eb60c3413b3a29ec3a3320037802e7debb9a90b6b157dee153abdc 2024-08-08T22:07:16.0399828Z deleted: sha256:4b54555217f7f889b4f5c43f115720f45963b8be4183e4c87c3b9e813cbf3929 2024-08-08T22:07:16.0400892Z deleted: sha256:e60bedd55ab6a7963172852bc199c80c39f3a3ea23354c41d8fd8bc6b141154a 2024-08-08T22:07:16.0401930Z deleted: sha256:cf97738287b3a0c55b67b024ad13970ee424e0b62e946249ad6180b95fcd8bc6 2024-08-08T22:07:16.0402993Z deleted: sha256:9e982b48aec3a7dbde4854e72ea1c3b3051b7b72deb76ed8872702ee27e0edbc 2024-08-08T22:07:16.0404072Z deleted: sha256:0ba0b263f6f6d1d9d189c7290599cba72cba9b3cda829505862aae8cb769c84a 2024-08-08T22:07:16.0405135Z deleted: sha256:66e9dd839f589fbb532d8ec0a630969713f0cade1718b1180f7fe26a538f3be0 2024-08-08T22:07:16.0406173Z deleted: sha256:c65a350f82fd0839046ab6b440122fb4d6f37fd9eed4122e1af8b1753e14fac5 2024-08-08T22:07:16.0407233Z deleted: sha256:d69dc134f0f0e1cee4240d762c4d5691b8094ba6f5e59eca754334dece3e4915 2024-08-08T22:07:16.0408309Z deleted: sha256:ae76f208c51d40eb54554ea1364e9af019accdef034fde105faaf175c4bdbd71 2024-08-08T22:07:16.0409365Z deleted: sha256:858f0f9520afafd43dfeb4c993a8d9a616dc8f02d22747a1d3b5d7197f652245 2024-08-08T22:07:16.0410565Z deleted: sha256:861076dfb9595f35cc788c7994492af10057765f7e1144a58677a8bea20c0090 2024-08-08T22:07:16.0411638Z deleted: sha256:2d71d4b9defc3248e0657b644d8fa8746329ee53252128df6260bdac07a1e601 2024-08-08T22:07:16.0412694Z deleted: sha256:29004ba29a25af35c37b4fc76b65cdd1a6e10e3dbd5fd02ed2d24fb47f45b7d6 2024-08-08T22:07:16.0413751Z deleted: sha256:b1411e2373c105437b4d99814f459a19ee299ef18c3a57e6a86e24206d1f90b9 2024-08-08T22:07:16.0414792Z deleted: sha256:32cc529ff369f7121a4e6bc51cdb7090d9c47d98233037c279730828347723ac 2024-08-08T22:07:16.0415854Z deleted: sha256:7235b067158b7aca2be16b9d2dbae06b610267aaeb41be289c77f2b801fd5e8c 2024-08-08T22:07:16.0416894Z deleted: sha256:f000b1b348e11f95e2e91541f128ca3e6695a00742fe6e16215bc321f213362c 2024-08-08T22:07:16.0417942Z deleted: sha256:88e9dd01ce433a853ac999c98231d6d47b4dd3a03391dfd56d4ea80528388089 2024-08-08T22:07:16.0419029Z deleted: sha256:cbdf41f558fd321dd6bdae94b52dcd65e7d35d6d3d7d822bcc3c58a1bb22872b 2024-08-08T22:07:16.0420183Z deleted: sha256:ac87fc04ff8d229dcfcb5d951627b7914cbc33c740161fc658608a75b41fd40a 2024-08-08T22:07:16.0421313Z deleted: sha256:9374806889af634777725ffcac8da3cec3fbada86d187ab4a81c59bad7e5b606 2024-08-08T22:07:16.0422397Z deleted: sha256:8512ea010bd0fbdceb7b671c3f6b2cced0fda4187080e2f1ff1c6e6037979dbb 2024-08-08T22:07:16.0423463Z deleted: sha256:14cbe137d139085b0f8df186eed7a8bbd9081b7c3b803e7b64365b0c3a2aee52 2024-08-08T22:07:16.0424529Z deleted: sha256:88ced829326eed4b4c235f0f806b6c686c1b2ef209dc319b7f03731c04bcf0d5 2024-08-08T22:07:16.0425599Z deleted: sha256:fa2f10a66c16fac3a2d5642d21f2cd69b0148111dd1c930dab83cb97ce6682cf 2024-08-08T22:07:16.0426672Z deleted: sha256:7f83eb696a42997925b1e29da8e78d8e2df08443db59a8585cdc72cd94c1c782 2024-08-08T22:07:16.0427693Z deleted: sha256:4dd8500484390881a49bd9ed71859da08379e801e124b7b57847742858857c68 2024-08-08T22:07:16.0428747Z deleted: sha256:f69f31dbdcdab61878bc9e8f2333afff7e73e3d07408f329c583f6d790c5ad07 2024-08-08T22:07:16.0429826Z deleted: sha256:9eb132174f81d00607875e3d95ac8be9c9fda5ed780de6df7efdacf14c33a568 2024-08-08T22:07:16.0430889Z deleted: sha256:24e7cb4074c44e2087d68312d0efafb74732b8999579a1c74bd9363e6495efb0 2024-08-08T22:07:16.0431955Z deleted: sha256:c6d54db7315b535608dbab74fbbf6679f7c0c228a45dac092f7ba7a3addc774f 2024-08-08T22:07:16.0433033Z deleted: sha256:feee1b527542c64f2f2ddfb22b32aea7bb209d132d82d50ea65b46829c26b31e 2024-08-08T22:07:16.0434148Z deleted: sha256:92d2aed80e97a958da3271d1da4cb80fef8b9bd257e5939d6b7ffea0f99cbcc8 2024-08-08T22:07:16.0435223Z deleted: sha256:983e83d0d3bf5b974d81434cd3a3deb5975c077e72dc7134f9c70a7f601255cd 2024-08-08T22:07:16.0436292Z deleted: sha256:433544e4353c9e0269dc71fa16c00ddcecdcd5676ed0df618c7b51ac78e23053 2024-08-08T22:07:16.0437379Z deleted: sha256:fc24af9fd5121adfbc7b75df0abb1ef8c2d3868f33b7b6b1e8bf4e214fac4bd6 2024-08-08T22:07:16.0438442Z deleted: sha256:ce1492ad94b71204f8b3fb8903d0ad48a96652635e7fc9347e430639ca15a737 2024-08-08T22:07:16.0439480Z deleted: sha256:96f2ae9435e6e12f68a88dc404c8079b2796639589970c34894a86ccff3f3732 2024-08-08T22:07:16.0440559Z deleted: sha256:5310d2bc0952e7c6d8cbade7ecafd88ce3a04d9da303fa6419583e13a02b58a9 2024-08-08T22:07:16.0441597Z deleted: sha256:099076d608a55a027d60f5b572a3991a8d690a01500acf2c696933a7e6d38650 2024-08-08T22:07:16.0442646Z deleted: sha256:862b05d0f8f7e3dff2d04bd49784a9901f4e22c5dd6544ae72eb3be05e23fffe 2024-08-08T22:07:16.0443717Z deleted: sha256:f7a888e5f7904827f6d71cebcba5169dbc7a78b20f228b9b9b625f38e0c52f24 2024-08-08T22:07:16.0444772Z deleted: sha256:ab9f75a4a7a636aae480f630eaddc05133dda64f1c42445b617c97d3d209991c 2024-08-08T22:07:16.0445860Z deleted: sha256:56dda0c9cee64db5c10d5dd5f08c6f6707263f5ee98e8a075d635e5be996e855 2024-08-08T22:07:16.0446948Z deleted: sha256:5faf9c0a9efe4675ecd21a4ec417d51077d5e75da9e673161a94e7d6cd43f92c 2024-08-08T22:07:16.0447593Z 2024-08-08T22:07:16.0447784Z Total reclaimed space: 25.47GB 2024-08-08T22:07:16.0532047Z Post job cleanup. 2024-08-08T22:07:16.0596732Z Post job cleanup. 2024-08-08T22:07:16.1800400Z [command]/usr/bin/git version 2024-08-08T22:07:16.1857678Z git version 2.40.1 2024-08-08T22:07:16.1920499Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/1d836005-bf07-47c4-9fbd-acf3bc2adf60' before making global git config changes 2024-08-08T22:07:16.1922105Z Adding repository directory to the temporary git global config as a safe directory 2024-08-08T22:07:16.1928077Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-08T22:07:16.1974877Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-08T22:07:16.2017272Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-08T22:07:16.2371052Z Entering 'android/libs/fbjni' 2024-08-08T22:07:16.2435284Z Entering 'third_party/FP16' 2024-08-08T22:07:16.2499909Z Entering 'third_party/FXdiv' 2024-08-08T22:07:16.2564542Z Entering 'third_party/NNPACK' 2024-08-08T22:07:16.2629648Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-08T22:07:16.2694016Z Entering 'third_party/XNNPACK' 2024-08-08T22:07:16.2778413Z Entering 'third_party/benchmark' 2024-08-08T22:07:16.2843717Z Entering 'third_party/cpp-httplib' 2024-08-08T22:07:16.2906785Z Entering 'third_party/cpuinfo' 2024-08-08T22:07:16.2974185Z Entering 'third_party/cudnn_frontend' 2024-08-08T22:07:16.3038481Z Entering 'third_party/cutlass' 2024-08-08T22:07:16.3111866Z Entering 'third_party/eigen' 2024-08-08T22:07:16.3178276Z Entering 'third_party/fbgemm' 2024-08-08T22:07:16.3243424Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-08T22:07:16.3306418Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-08T22:07:16.3371129Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-08T22:07:16.3440712Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-08T22:07:16.3504142Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-08T22:07:16.3570670Z Entering 'third_party/flatbuffers' 2024-08-08T22:07:16.3636973Z Entering 'third_party/fmt' 2024-08-08T22:07:16.3703202Z Entering 'third_party/foxi' 2024-08-08T22:07:16.3767440Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-08T22:07:16.3832152Z Entering 'third_party/gloo' 2024-08-08T22:07:16.3897504Z Entering 'third_party/googletest' 2024-08-08T22:07:16.3962388Z Entering 'third_party/ideep' 2024-08-08T22:07:16.4024824Z Entering 'third_party/ideep/mkl-dnn' 2024-08-08T22:07:16.4098258Z Entering 'third_party/ittapi' 2024-08-08T22:07:16.4162459Z Entering 'third_party/kineto' 2024-08-08T22:07:16.4226820Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-08T22:07:16.4290926Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-08T22:07:16.4355727Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-08T22:07:16.4419179Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-08T22:07:16.4483625Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-08T22:07:16.4546412Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-08T22:07:16.4613039Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-08T22:07:16.4676905Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-08T22:07:16.4740211Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-08T22:07:16.4805567Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-08T22:07:16.4871053Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-08T22:07:16.4934412Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-08T22:07:16.5001411Z Entering 'third_party/mimalloc' 2024-08-08T22:07:16.5065744Z Entering 'third_party/nccl/nccl' 2024-08-08T22:07:16.5129823Z Entering 'third_party/nlohmann' 2024-08-08T22:07:16.5195273Z Entering 'third_party/onnx' 2024-08-08T22:07:16.5277084Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-08T22:07:16.5340838Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-08T22:07:16.5407498Z Entering 'third_party/opentelemetry-cpp' 2024-08-08T22:07:16.5473261Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-08T22:07:16.5535927Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-08T22:07:16.5605185Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-08T22:07:16.5668486Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-08T22:07:16.5732595Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-08T22:07:16.5796303Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-08T22:07:16.5858890Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-08T22:07:16.5921091Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-08T22:07:16.5988225Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-08T22:07:16.6053404Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-08T22:07:16.6142020Z Entering 'third_party/pocketfft' 2024-08-08T22:07:16.6205827Z Entering 'third_party/protobuf' 2024-08-08T22:07:16.6273064Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-08T22:07:16.6336609Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-08T22:07:16.6402196Z Entering 'third_party/psimd' 2024-08-08T22:07:16.6465705Z Entering 'third_party/pthreadpool' 2024-08-08T22:07:16.6530773Z Entering 'third_party/pybind11' 2024-08-08T22:07:16.6595447Z Entering 'third_party/python-peachpy' 2024-08-08T22:07:16.6658830Z Entering 'third_party/sleef' 2024-08-08T22:07:16.6722281Z Entering 'third_party/tensorpipe' 2024-08-08T22:07:16.6787411Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-08T22:07:16.6852090Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-08T22:07:16.6915244Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-08T22:07:16.6978864Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-08T22:07:16.7041041Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-08T22:07:16.7128457Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-08T22:07:16.7160374Z http.https://github.com/.extraheader 2024-08-08T22:07:16.7174430Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-08T22:07:16.7219535Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-08T22:07:16.7570906Z Entering 'android/libs/fbjni' 2024-08-08T22:07:16.7614351Z http.https://github.com/.extraheader 2024-08-08T22:07:16.7654562Z Entering 'third_party/FP16' 2024-08-08T22:07:16.7699169Z http.https://github.com/.extraheader 2024-08-08T22:07:16.7739866Z Entering 'third_party/FXdiv' 2024-08-08T22:07:16.7784600Z http.https://github.com/.extraheader 2024-08-08T22:07:16.7823352Z Entering 'third_party/NNPACK' 2024-08-08T22:07:16.7867386Z http.https://github.com/.extraheader 2024-08-08T22:07:16.7907699Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-08T22:07:16.7950958Z http.https://github.com/.extraheader 2024-08-08T22:07:16.7991089Z Entering 'third_party/XNNPACK' 2024-08-08T22:07:16.8035742Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8094723Z Entering 'third_party/benchmark' 2024-08-08T22:07:16.8138305Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8178609Z Entering 'third_party/cpp-httplib' 2024-08-08T22:07:16.8221862Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8261829Z Entering 'third_party/cpuinfo' 2024-08-08T22:07:16.8305953Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8345048Z Entering 'third_party/cudnn_frontend' 2024-08-08T22:07:16.8389467Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8430414Z Entering 'third_party/cutlass' 2024-08-08T22:07:16.8474206Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8522838Z Entering 'third_party/eigen' 2024-08-08T22:07:16.8566258Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8608938Z Entering 'third_party/fbgemm' 2024-08-08T22:07:16.8653097Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8694042Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-08T22:07:16.8736828Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8776588Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-08T22:07:16.8820180Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8860632Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-08T22:07:16.8903069Z http.https://github.com/.extraheader 2024-08-08T22:07:16.8949655Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-08T22:07:16.8994158Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9034209Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-08T22:07:16.9079146Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9120320Z Entering 'third_party/flatbuffers' 2024-08-08T22:07:16.9164144Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9207530Z Entering 'third_party/fmt' 2024-08-08T22:07:16.9251252Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9291692Z Entering 'third_party/foxi' 2024-08-08T22:07:16.9336213Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9377161Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-08T22:07:16.9420469Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9460915Z Entering 'third_party/gloo' 2024-08-08T22:07:16.9504629Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9543280Z Entering 'third_party/googletest' 2024-08-08T22:07:16.9587643Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9627815Z Entering 'third_party/ideep' 2024-08-08T22:07:16.9672301Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9710918Z Entering 'third_party/ideep/mkl-dnn' 2024-08-08T22:07:16.9754944Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9804494Z Entering 'third_party/ittapi' 2024-08-08T22:07:16.9847071Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9888211Z Entering 'third_party/kineto' 2024-08-08T22:07:16.9931322Z http.https://github.com/.extraheader 2024-08-08T22:07:16.9972679Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-08T22:07:17.0016644Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0057018Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-08T22:07:17.0101097Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0142835Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-08T22:07:17.0186675Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0226915Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-08T22:07:17.0270392Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0310760Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-08T22:07:17.0356022Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0396358Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-08T22:07:17.0439554Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0481725Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-08T22:07:17.0524824Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0566611Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-08T22:07:17.0609533Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0649684Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-08T22:07:17.0694520Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0735625Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-08T22:07:17.0778952Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0821123Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-08T22:07:17.0864855Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0903698Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-08T22:07:17.0947217Z http.https://github.com/.extraheader 2024-08-08T22:07:17.0990240Z Entering 'third_party/mimalloc' 2024-08-08T22:07:17.1033444Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1074153Z Entering 'third_party/nccl/nccl' 2024-08-08T22:07:17.1117545Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1157101Z Entering 'third_party/nlohmann' 2024-08-08T22:07:17.1201437Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1242666Z Entering 'third_party/onnx' 2024-08-08T22:07:17.1286519Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1344038Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-08T22:07:17.1387792Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1427645Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-08T22:07:17.1473330Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1515351Z Entering 'third_party/opentelemetry-cpp' 2024-08-08T22:07:17.1559503Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1601609Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-08T22:07:17.1643971Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1683209Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-08T22:07:17.1726071Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1767589Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-08T22:07:17.1810246Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1849180Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-08T22:07:17.1892909Z http.https://github.com/.extraheader 2024-08-08T22:07:17.1933206Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-08T22:07:17.1976253Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2015003Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-08T22:07:17.2058840Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2097773Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-08T22:07:17.2140944Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2181648Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-08T22:07:17.2223737Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2266397Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-08T22:07:17.2309129Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2350680Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-08T22:07:17.2394553Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2456590Z Entering 'third_party/pocketfft' 2024-08-08T22:07:17.2500978Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2541368Z Entering 'third_party/protobuf' 2024-08-08T22:07:17.2584609Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2628176Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-08T22:07:17.2671720Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2710084Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-08T22:07:17.2752432Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2794122Z Entering 'third_party/psimd' 2024-08-08T22:07:17.2838007Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2876568Z Entering 'third_party/pthreadpool' 2024-08-08T22:07:17.2920014Z http.https://github.com/.extraheader 2024-08-08T22:07:17.2958771Z Entering 'third_party/pybind11' 2024-08-08T22:07:17.3002607Z http.https://github.com/.extraheader 2024-08-08T22:07:17.3041167Z Entering 'third_party/python-peachpy' 2024-08-08T22:07:17.3086885Z http.https://github.com/.extraheader 2024-08-08T22:07:17.3125601Z Entering 'third_party/sleef' 2024-08-08T22:07:17.3170060Z http.https://github.com/.extraheader 2024-08-08T22:07:17.3209893Z Entering 'third_party/tensorpipe' 2024-08-08T22:07:17.3253152Z http.https://github.com/.extraheader 2024-08-08T22:07:17.3293928Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-08T22:07:17.3336113Z http.https://github.com/.extraheader 2024-08-08T22:07:17.3376385Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-08T22:07:17.3418551Z http.https://github.com/.extraheader 2024-08-08T22:07:17.3457050Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-08T22:07:17.3500023Z http.https://github.com/.extraheader 2024-08-08T22:07:17.3538926Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-08T22:07:17.3581674Z http.https://github.com/.extraheader 2024-08-08T22:07:17.3620992Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-08T22:07:17.3665189Z http.https://github.com/.extraheader 2024-08-08T22:07:17.3815934Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-08T22:07:17.3837679Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-08T22:07:17.3844359Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-08T22:07:17.3845064Z ##[endgroup] 2024-08-08T22:07:25.8884875Z Cleaning up orphan processes